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FOREWORD 



An examination of the educational data base reveals relatively few 
citations on protocol materials. A very large number of ERIC-processed 
documents indexed under "protocol materials*' are primarily about micro- 
teaching. Undoubtedly there are some documents which hcve been circulated 
among those actively producing protocol materials, but, for whatever 
reason, materials on protocols available to the general educational 
community are limited. The Clearinghouse is pleased to present a series 
of papers which provide some suggestions on how to move from a con- 
ceptualization on integrating theory and practice; to production, 
field testing, and revising; and finally to publicizing and marketing. 

Special credit is due Donald Cruickshank for his help in 
conceptualizing the contents of this publication. Recognition is also 
due to the writers who contributed their articles. 

It is appropriate to note the unique role played by Lawrence Lipsitz, 
editor of Educational Technology, The possibility that this series of 
articles would be published in that journal provided a stimulus to each 
of us. The ERIC system benefits from the cooperation of editors 
who provide a valued supplement to the dissemination capabilities 
of the ERIC Document Reproduction Service. 

It is hoped that this document will provide a solid addition to 
the educational literature on an important topic: the creation of 
training materials which have considerable potential for transforming 
the abstract study of education into something more vital, dynamic, 
and meaningful. When this happens, it will be a just reward for the 
inventor of the protocol idea in education — B. Othanel Smith. 

You may do further research on this topic by checking issues of 
Research in Education (RIE) and Current Index to JouFnals in Education 
(CUE). Both RIE and CUE use the s ime descriptors (index terms). 
Documents in RIE are listed in blocks according to the clearinghouse 
code letters which processed them, beginning with the ERIC Clearinghouse 
on Career Education (CE) and ending with the ERIC Clearinghouse on the 
Disadvantaged (UD) . The clearinghouse code letters, which are listed at 
the beginning of RIE, appear opposite the ED number at the beginning of 
each entry. *»SF' (School Personnel) designates documents processed by 
the ERIC Clearinghouse on Teacher Education. For readers uncertain 
how to use ERIC capabilities effectively, we recommend How To Conduct 
a Search Through ERIC , ED 036 499, microfiche $.75; hardcopy $1.85, 
It is available from the ERIC Document Reproduction Service, P. 0. 
Box 190, Arlington, Virginia 22210. 

— Joel L. Burd in , Director 

February 1974 
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ABSTRACT 



This publication brings together seven papers by writers who 
nave been extensively involved in the preparation and use of protocol 
iTiateriais in teacher education. These papers are a) Protocol Materials 
Historical Notes on Protocols Development, by Doris V. Gunderson; 

b) The Protocol Materials Movement: An Exemplar of Efforts To 

Wed Theory and Practice in Teacher Education, by uonald R. Cruickshank; 

c) The University of Colorado Protocol Project: A Case Study, by 
Celeste P. Woodley and Laura A. Driscoll; d) A Catalogue of Concepts 
in the Pedagogical Deniain of Teacher Education, by Bryce B. Hudgins; 
e) The Protocol Materials Program, by Donald H. Orlosky; f) A 
Protocol Materials Evaluation*! The Language of Children, by Victor 
M. Rentel; and g) A Surve ' of Protocol Materials Evaluation, by 
John E. Cooper. OIBM) 



To expand 3 bibliography using ERIC, descriptors or search terms 
are used. To use a descriptor: (1) Look up the desc-^iptor in the 
SUBJECT JNDEX of monthly, semi-annual, or annual issue of Research in 
Education (RIE), (2) Beneath the descriptors you will find title (s) 
of documents. Decide which title (s) you wish to pursue. (3) Note the 
**ED" number beside the title. (4) Look up the "ED" number in the 
"DOCUMENT RESUME SECTION" of the appropriate issue of RIE. With the 
nurnber you v;ill find a summary of the document and often the document's 
cost in microfiche and/ or hardcopy. (5) Repeat the above procedure, 
if desired, fox other issues of RIE and for other descriptors. (6) For 
information about how to order ERIC documents, turn to tne back pages 
of RIE. (7] Indexes and annotations of journal articles, can be found 
in Cu rrent Index xo Journals in Education by following the same pro- 
cedure. Periodical articles cannot be secured through ERIC. 

TOPIC: Protocol Materials: Training Materials For Uniting Theory and 



ERIC DESCRIPTORS 



Practice 



DESCRIPTORS TO USE IN CONTINUINC, SEARCH OF RIE AND CUE: 



* Behavioral Objectives 

* Program Development 
*Program Evaluation 



^Protocol Materials 
*Teacher Behavior 
*Teacher Education 



*AsterLsk(sJ indicate major descriptors. 



THEME INTRODUCTION 
Protocols: A Reality Approach to 
Vitalizing Teacher Education 



Since Smith introduced the term **protocols" into the literature,* 
there has been a continuing search for the meaning and use of such 
materials. Protocol materials are commonly recognized a^ a high- 
potential approach to vi'^'ilizing pre- and in-service teacher education. 
These materials supplement others used in teacher education and provide 
a creative way to meld theory and practice into effective learning. 

The literature on protocol materials tends to be somewhat limited. 
The authors of these papers hope that they will increase understanding 
of an important concept, encourage experiment with effective use, and 
stimulate assessment of that use and continual efforts to improve protocols 
utili zation. 

Gunderson delineates the need for developing understandings about 
learning and gives an overview of how protocols conceptualization and 
development were stimulated by U. S. Office of Education (USOE) efforts. 
Her perspective is that of a foimer USOE project monitor for the 
protocols project. 

Cruickshank defines protocols as an original record of an event 
of educational significance, utilized in order to permit learners to 
interpret the event or solve the problem depicted in che event, with 
appropriate concepts from related fields of study. For clarity, he 
explains the term "events of educational significance'* and distinguishes 
between protocols and field-based education. He discusses very briefly 
the development of protocols, then examines how the initiation of USOE 
support modified the original definition. (The USOE Master Coordinate 
System is appended.) Finally he questions whether the powerful notion 
of protocols as originally conceived will become lost as the individual 
projects move toward a less empirically based,^ more prescriptive concept. 

Woodley and Driscoll describe the University of Colorado Protoce ' 
Project and show how an educational idea is translated . into an instruc- 
tional product. They detail the steps taken in developing, testing, 
and disseminating the protocol material s relevant to teacher-pupil 
classroom interaction and share the insights, methods, problems, 
mistakes, and successes of their project. 

Hudgins responds to the need felt since the beginning of the protocol 
materials program for a conceptual map of the pedagogical domain of teacher 
education. This should aid in deciding which concepts, or at least which 
group of concepts, should be developed into protocols and should have 
two functions: (a) to Identify appropriate concepts in the literature 
of teacher education and (b) to give salient and significant information 
to help portray the concept. Such a map has been under development under 
the direction of the author. While it features concepts pertaining to inter- 
active teaching, the author recognizes that other concepts of teacher 
education will require future development. An extensive example of 
seeking out and • classi fy ing one concept is given, the for^nat of the 
catalogue Is explained, p.nd samples from the catalogue are shown. 

* B. Othanel Smith and others. Teachers for ^he [Real World (Washington, 
D.C.: American Association of Colleges for Teacher Education, 1969) 
pp. 52 -53, 62-64, .and 158. 



Orlosky provides a history of the protocol materials and then 
discusses major developments, including the definition of protocols, 
developipent procedures, selection of concepts, protocols vs. observation, 
the master plan, the technical and educational criteria for the protocols, 
supplementary materials included in the packages, field testing, and 
dissemination. He concludes with a consideration of the current status 
of the program and summarizes the possible future for protocols. 

Rentel analyzes protocol materials developed by Ohio State University 
College of Education to illustrate concepts of oral language. The author 
describes how the concepts were selected and briefly defines those 
developed as protocols. Me discusses the method and procedure for revising 
the materials, based on the results of a questionnaire answered by those 
involved in the revisions. The author discusses the recommendations 
made by participants and instructors. In conclusion, he describes the 
method, procedure, instruments, and results of the product evaluation 
and discusses their meaning. 

Cooper summarizes evidence of the effectiveness of protocol materials 
prepared in a number of institutions, including Utah State University, 
Michigan State University, the Far West Laboratory for Educational 
Research and Development, Indiana University, and Southern Illinois 
University at Edwardsvi lie. He places this evaluation in che context 
of a brief review of protocol materials and concludes with some 
recommendations for the future. 
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Protocol Materials: Historical Notes on 
Protocols Development^ 



hy Doris V, Gunderson 
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Universities and . school systems, the inst^utions that train edu- 
cational personnel, recognize the need for-TlTaLterial that will help 
teachers acquire the understanding and skills necessary to teaching. 
Although innumerable written materials exist in education and allied 
fields, little or no evidence indicates that, these materials' are tools 
capable of inducing desired behaviors in pre- or in-service teachers. 

Traditionally, the teacher's primary function at r.ny tevel has 
been to help students learn. If teachers are to help students learn, 
they must know how to teach so 'that teachers will learn. Teachers 
must know th'e subject matter, and they must know how to teach. The 
teacher must understand the student and understand the learning pro- 
cess. Individuals differ in learning rates and respond to teaching- 
learni.ig situations . in various ways. 

?' 

Developing understai"jdings about learning is not easy. Extensive 
observation of students is essential, yet observation without focus 
accomplishes little. The observation must be directed; the teacher 
must look for particular kinds of behaviors. There is no guarantee 
that a given stimulus will result in the behavior the teacher wishes 
to observe. Even if a particular behavior is exhibited, the instance 
may be fleeting, and the observer has only his recollection for con- 
sidered study. 

The observer's preparation for interpret in g behavio r usually 
consists of little more than courses presented in the traditional 
reading- lecture-discussion manner.' The instruct ion* i s divorced from 
reality. One"^ means of bridging the gap between theory and reality 
is to reproduce a variety of behaviors of students, teachers, and 
others in a permanently recallable form. A particular segment of 
behavior can thus be produced again and again to be studied, analyzed, 
and the concepts appropriate to its interpretation spotlighted, ex- 
plained, learned, and reviewed. The concepts can be applied to the 
understanding of other behaviors. Instructional materials of this 
type are referred to as protocol materials. 

Protocol materials constitute one of two general categories of 
instructional materials for teachers, namely;^^ the materials that direct 
the pre- or in-service teacher in studying his and others' behavior. 
Materials that guide him in systematic practice of the skills he -must 
acquire are called training materials and are designed to help the 
teacher in the acquisition of skills. They provide for (a) identi- 
fication of skills, (b) description of behavior entailed by the skills, 
(c) performance of the behavior, and (d) feedback to the performer and 
further performance by him. 

Cognitions are developed primarily through the use of protocol 
materials which provide (a) segments of behavior categorized for 
the purpose of teaching concepts and principles used in interpreting 
behavior as well as the social context in which the teacher works,* (b) 
segments of behavior categorized for the purpose of teaching knowledge 
about knowledge, and (c) segments of behavior categorized for the 
purpose of teaching self-understanding. 
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Protocol materials are designed to bridge the gap between theory 
and the teaching-learning situation. Since they provide reproductions 
of behavior, they foster the teacher's interpretive and diagnostic 
competency. 

A basic problem in prieparing protocol materials is precision in 
defining a concept; a colicept must be defined exactly and its attributes 
specified in terms which permit no misunderstanding. Concepts, according 
to Carroll, are properties of organismic experience, the ^'abstracted and 
often cognitively structured classes of mental experience learned by 
organisms in the course, of their life histories." As the complexity of 
the concepts increases, the necessity for an appropriate sequencing of 
positive and negative .instances to assure adequate learning cr the con- 
cept becomes greater.^ 

, < 

Recognizing ;:he demand for materials for teacV.ing concepts in 
teacher education programs, the Bureau' of Educ?> ^ ion al Personnel Devel- 
opment (USOE) in 1970 initiated an effort tr train educational personnel 
to develop and use protocol materials, ^'^terials developed for training 
teachers generally have been preparer^ in isolation with no field testing 
during the development stages. ice protocol materials should be 
attuned to the problems teach :. s encounter, the Office of Education 
insisted that people shou.^.d oe trained to develop protocol materials, 
with field testing and ^edification based on the field testing an inte- 
gral part of the trn* iing program. In addition to the field testing 
carried out by eac'i project director, the Florida State Department of 
Education, under a grant from the Office of Education, is conducting 
field tests of protocol materials developed in the various projects with 
pre- and in-service teachers in school systems and institutions of 
higher education in Florida. 

The protocol materials effort is essentially a training program. 
Project directors are trained to develop and use protocol materials./ 
The effort is under the direction of the Leadership Training Institute, 
a group of consultants outsiue the Office of Education, headed by B. 
Othanel Smith of the University of South Florida. The group is respon- 
sible for providing technical assistance to the project directors. The 
training' involves several stages'. The concepts to be exemplified in 
protocol materials must be selected and analyzed. The concepts must' 
be critical to teacher education; that is, they must be concepts that 
teachers need to know. A paramount consideration is utility; the con- 
cepts to be portrayed in the materials will be useful at any institution 
or agency which trains or retrains teachers. 

The materials being produced, primarily films, cover diverse 
subject matter areas such as educational psychology, social psychology, 
reading, li-terature, language acquisition. Black English, teaching 
analysis, and social studies. The important result of this project 
to the future of teacher training lies in thp fact that these leaders 
in the field of teacher education will themselves have acquired skills 
in the development of protocol materials. 



NOTES 



1. John B. Carroll, "Words, Meanings and Concepts, *V Harvard Educational 
Review , September 1964, pp. 178-2Q2. 
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The Protocol Materials Movement: An Exemplar of 
Efforts To Wed Theory and Practice in Teacher 

Education 



jby Donald R. Crulckshank 



PROTOCOLS 

ft 

My. initial encounter with the term "protocol" in an educational 
context was confusing since I knew it only in the limited sense of 
describing a rigid, long-established deferential code practiced in 
government, especially diplomatic circles. Consequently, I brought an 
unsuitable meaning to the printed page which had to be corrected. In 
the sense in which it is used herein, at least initially, a protocol 
is an original record of an event of educational significance. 

The rationale for creating and using protocols (or protocol mate- 
rials) probably was first brought to the attention of the educational 
community at large by Smith and others.! This group inquired into the 
education of disadvantaged youth and teacher education and subsequently 
made several recommendations and proposals calling for change in the 
latter. Their report states: 

Teachers fail because they have not been trained calmly to analyze 
new situations against a firm background of relevant theory. Typ- 
ically they base their interpretations of behavior on intuition 
and common sense. ... If the teacher is incapable of understand- 
ing classroom situations, the actions he takes will often increase 
his difficulties. 2 

This initial assertion that teachers need to be trained to analyze 
classroom situations against a firm background of relevant theory is 
amplified and supported as theoretical knowledge is defined,^ and it 
^ is noted that such knowledge is becoming more abundant in the behavioral 
sciences. 4 Further, the Smith task force asserts that "the basic ele- 
ments of theoretical knowledge are concepts. "5 Now we can begin to 
operationalize the earlier definition of protocols from an original 
record of an event of educational significance to an original record 
of an event of educational significance utilized in order to permit 
learners to Interpret the event or solve the probl&r. depicted in the 
ev^nt using appropriate concepts from related fields of study Including 
psychology^ sociology, anthropology , philosophy , or others. 

It seems clear from the extended definition that at least two 
decisions are paramount for the developer and/or user of protocols. 
First, he must answer the question, "What is an event of educational 
significance?" Second, he must determine which available concepts are 
likely to increase the subject's capacity to interpret and understand 
the event recorded in the protocol. This one-two procedure is amplified 
elsewhere^ and referred to as the "situation- first strategy." 

Events of Educational Signi ficance 

A 

Smith equates an event of educational significance with situations 
the teacher most frequently encounters but notes that since no systematic 
analysis of the teacher's frequent work exists, the event must be 
selected from practical experience. ' The followini; catet^orics of 
events are suggested: 

I . C 1 as sr oom s it ua t i oh s 
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a. Instructional situations 

b. Situations of classroom management and control 
2. Extraclassroom situations 

a. Situations that arise in planning school programs, 
working with peers and the administration 

b. Situations that occur in working with parents and other 
members of the community 

c. Situations that occur in working in professional organi- 
zations. 8 

Following this '^situations- first strategy," once events of educa- 
tional significance are determined, they must be inspected to consider 
what theoretical not ions--concepts — are needed to understand and inter- 
pret them. Indeed, Smith cautions, "No instruction should begin until 
the instructor has studied the [event] with extreme caie."9 

The following will help to clarify. It describes an example of 
what inner-city teachers consider to be frequent and bothersome class- 
room behavior, one criterion which could be applied in selecting events 
of educational significance. It probably falls in category lb above. 

The teacher has broken the sixth grade class into small groups 
each of which is experimenting with a pendulum. In one of the 
groups, one student is swinging the pendulum, another is keeping 
time using a stopwatch while a third is recording the time. 
Others are in the group but these three are prominent. During 
an interval between timing the period of the pendulum the 
recorder, Wesley, reaches over and abruptly takes the stopwatch 
from that student using it, Bradley. The teacher mildly admon- 
ishes Wesley and orders him to return the watch. Wesley does 
so reluctantly and the experiment continues. A child from an- 
other group approaches the teacher and a conversation ensues. 
Soon scuffling noises occur and a fight breaks out between 
Bradley and Wesley. The teacher intervenes verbally and the 
physical aggression changes to name-calling and accusations. 
Bradley holds up his watch which is broken and claims that 
Wesley is responsible. Wesley responds that "It was an accident." 
Bradley counters callirg Wesley a liar. Wesley charges that 
Bradley stole the watch anyway and tells Bradley to just steal 
another one as the incident closes. ■'■0 

If the above incident were used as a protocol, it follows that the 
teacher, alone or with the assistance of colleagues from the behavioral 
sciences, must inspect the event and consider what theoretical notions 
— concepts--are needed to understand and interpret it. To begin, one 
might identify or describe the phenomena observed. These seem to in- 
clude sixth- grade class, grouping for instruction, experiment, pendulum, 
differentiation of labor, , stopwatch, keeping time, recording, seizure, 
disapproval-repriir.and, acquiescence, interruption, fight in g, inter- 
vention, name-calling, accusation, destruction, informing-tatt ling, 
and so forth. It is significant that the teacher educator be able to 
help students select from among these phenomena those which may be of 
special interest and power in explaining, understanding, and inter- 
preting the event. Which phenomena and/or related concepts do you see 
as most germane? 



Up to this point it is clear that the protocol movement is con- 
cerned with finding out what teachers need to know in order to function 
more effectively in educationally significant situations. This is 
quite a departure from the usual method of teacher education which 
exposes the undergraduate to knowledge organized seemingly logically 
into courses in education! The usual method requires the learner to 
apply theory, explaining human behavior, for example, to the classroom. 
Conversely, the protocol way forces the teacher educator to begin with 
classroom events and show how certain concepts make those events more 
understandable. 

Protocols and/versus Field-based Education 

Since recently there has been a big push toward field-based teacher 
education, a word of caution may be in order. Promoters of field-based 
programs are convinced that you learn about teaching by teaching and so 
naturally professional education should occur where teaching takes 
place — in the schools. The Smith task force makes this notion suspect, 
stating: 

Teaching behavior is complex, involving interactions with both 
pupils and materials of instruction. It cannot be studied in the 
classroom because behavior perishes as it happens and nothing is 
left to analyze except the memory or a check sheet. The fidelity 
of the memory is questionable and not detailed enough. The infor- 
mation contained on check sheets is almost no record at all. To 
learn to interpret situations they must be held Jji situ or repro- 
duced at will approximately as they occurred. It is then possible 
to study the situations at length and use concepts ... to 
interpret them. 

If the protocol movement^-an exemplar of theory into practice — is 
tenable, it behooves the field-based movement to explain how it is, or 
will be, taken into account therein. Quality of experience in a class- 
room may not be an adequate replacement for the kind of quality of 
experience the study of protocols could provide. 

Development of Protocols 

What is the progression of steps suggested in developing a proto- 
col program? As mentioned earlier, the Smith report equates an event 
of educational significance with situations the teacher most frequently 
encounters. Consequently '*the first step in the formulation of a 
[protocol materials program] is to identify the most general categories 
of situations teachers face."^^ These situations become the objects 
of study and are recorded in some manner. They are then placed in pre- 
determined categories such as classroom instructional situations, 
classroom management situations, extraciassroom situations, or others. 
Next, the situations are studied intensely to determine what theoretical 
knowledge (especially concepts) is required in order to understand and 
interpret the phenomenon displayed in the situation. Final)/, the pro- 
tocols are arranged in courses of instruction.^^ 
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SUPPORT AND REDIRECTION VIA WASHINGTON 



The Smith task force recognized that building new teacher educa- 
tion programs based upon protocols would be a massive and expensive 
undertaking. It therefore recommended that **a number of institutions, 
federal and state agencies, and professional organizations could co- 
operate" in the endeavor. 14 

Since the U.S. Office of Education through the National Defense 
Education Act (NDEA) had financed the work of the Smith task force, it 
came as no great surprise that it would be interested in considering 
and supporting selected task force recommendations. 

That support manifested itself in April 1970 when the former 
Bureau of Educational Personnel Development invited several selected 
institutions to submit proposals for the development of protocol mate- 
rials. With USOE intervention and support, the protocol movement was 
given sanction by Washington and thereby given prominence for a large 
section of the education community which until that time probably was 
not well acquainted with the S^ith report and its recommendations. 
(See ''The Protocol Materials Program'* by Orlosky, p. 69 for details 
of Washington's involvement. Herein, attention will be given only to 
the request for proposals (RFP) vrfiich was the universities' official 
invitation to partake in the protocol movement.) 

Guidelines 

The RFP and especially the accompanying guidelines were to rede- 
fine the original notion of protocols and redirect materials development. 
Most agree that the redefined notion of protocols and the new direction 
were laudatory. In fact, recipients of protocol grants used less of the 
Smith report and more of the Washington guidelines in their work. A 
few argued and still maintain that although the Washington approach was 
acceptable and appropriate. Smith's original notion of protocols as 
exemplars of classroom situation-based teacher education has never been 
understood fully and certainly has not been implemented in the Wash- 
ington-directed movemcriC. 

The RFP cover letter from Don Davies, then Associate Commissioner 
of Education, made clear (a) that USOE was supporting the notion of 
protocols per the Smith task force, (h) that the 40 persons receiving 
requests for proposals had been recommended by an ad hoc advisory 
committee, (c) that not more than a half million dollars of USOE first- 
year money would be involved, and (d) that the USOE protocol materials 
program was an "attempt to prepare and use such materials in order to 
help make the training of educational personnel more exact and real." 
Of the four enclosures accompanying Davies' letter. Attachment A which 
explained protocol materials and delineated the guidelines for sub- 
mitting proposals is probably most germane to the program's history. 

In Attachment A the following points were made: 



1. Although teachers in training take courses intended to help them 



interpret school-rp' ated phenomenon, this instruction is not 
meaningful nor a- successful as it should be because typically 
it occurs divorced from real ivy. 

2. As presently constituted, the brie/ periods of obseirvation , par- 
ticipation, or other laboratory expe/^ence do not cause teachers 

in training to see the relationship o£ educational theory and class- 
room practice. 

3. Educational technology is now available which would permit the 
reproduction of school-related events which c;?n be used in teacher 
education programs wherein appropriate theory can be learned and 
its relationship to reality seen. (For example, a particular 
segment of classroom behavior can be reproduced and, as it is 
viewed, made more understandable through the study of specific 
related concepts. ) 

Further, a distinction was drawn between protocol materials used to 
interpret and understand classroom life and training materials intended 
to teach the prospective teacher skills. 

The guidelines permitted significant alteration of Smith's defini- 
tion for protocols, eliminating the requirement that proposals be 
"original records" and permitting . them to be staged representations of 
reality,; more precisely, simulations, although the term is not mentioned. 
Henceforth, the concept of protocol could be attached to any authentic 
representation of an educationally significant event which is utilized 
In order to permit l.earners to interpret or solve the problem depicted 
using appropriate concepts from related fields of study. Clearly the 
emphasis turned more toward interpreting the event rather than capturing 
valid and correct ones, a problem which has eluded persons interested 
in research on teaching for some time. 15 Consequently, as pointed out 
elsewhere, already existing simulations or other forms of teaching mate- 
rials which are used to increase the learner's analytic and interpretive 
powers through the use of theoretical knowledge suddenly became proto- 
cols, and the newer concept, seemed to lose some of its newness and 
punch. 16 

The Master Coordinate System 

The guidelines further contained an intricate Master Coordinate 
System (see Appendix A) which was to be used to "develop protocol mate- 
rials in an orderly way.'* In contrast to the Smith report wherein 
protocols were defined jis original records of events of educational 
significance which would be made more understandable by the application 
of theory/ concepts to them, the Washington guidelines and the Master 
Coordinate System operationally defined protocols as illustrations of 
concepts. (See "The University of Colorado Protocol Project: A Case 
Study" p. 56 and "A Protocol Materials Evaliiation: The Language of 
Children" p. 83.) The Woodley group chose to illustrate concepts in 
the "pedagogical domain" of the Master Coordinate System. The Rcntel 
i^roup on the other hand worked in the "basic fields oP knowledge." So, 
following the guidelines, "original reooi'ds" were replaced by "authentic 
ones" and "events of educational signfi> anct ' by "concepts of educational 
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significance." (Hudgins, on p. 69 describes how he is attempting to 
prepare a catalog of the latter, ) Naturally the concepts ipust be 
selected with some educational event in mind but that event is given 
short shrift. Some reviewers of the protocol movement, as this one, 
fault it for failing to identify its proposed teacher education cur- 
riculum (concepts) after a careful analysis of the real world of 
teaching (educationally significant events). Instead, in a manner not 
well-suited to pedagogy, protocol developers were asked to behave as 
if there were a thoroughly explicated discipline of education vdiich 
presented concepts or ideas of major magnitude and accepted validity. 1^ 
All the developer had to do was to select an ornament (concept) from the 
Christmas tree (education discipline). Naturally only the most "illumi- 
nating" would be chosen. What happened in most cases was that developers 
fell back to looking through related disciplines for concepts. In the 
usual tradition^ concepts were lifted from psychology, for example, 
and many protocol projects simply became projects to develop visual 
materials which illustrate concepts from the behavioral sciences. The 
concepts became the ends rather than the means, primary rather than 
secondary, as the Smith task force seemed to advocate. 

Concept Overkill? 

Further, many projects seemed to forget that concepts often can 
be gained very quickly through definition. Projects seemed to run 
amok visually illustrating concepts. Concept overkill may have re- 
sulted. The above would suggest that although the Smith task force 
(directly or inadvertently) argued for the study of school life and 
the selection or development of theory to understand it, the protocol 
movement resorted to looking for some "neat" concepts which could be 
made visual and "colorful" via a protocol. Theory building and theory 
relating simply have not reached their potential yet in the movement. 

On June 1, 1970 the winners of the protocol competition were an- 
nounced in a letter to members of Congress, and eleven projects in ten 
states were funded for the first year. 

The projects were to be overseen by a Leadership Training Insti- 
tute (LTI)IS led by Smith. Orlosky (p.XXX) describes LTI, its member 
ship, and function. A thorough description of the individual protocol 
projects and the materials produced through 1971 is found in Kincaid.^^ 



A MAGNIFICANT OBSESSION? 

Probably viien the protocol program is recorded in American educa- 
tional history, it will be remembered as a magnificant attempt (perhaps 
an obsession) to relate theory to practice. Since at least another 
year of federal sponsorship remains, it is not appropriate to consider 
whether or not the movement will achieve this hoped-for result. At 
this writing, however, several things are becoming clearer and are 
worthy of comment. Individual protocol projects have moved away from 
the situation- first, analytic, interpretive strategy suggested by the 
Smith task force toward a less empirically based and more prescriptive 



concept- first approach based upon the M-aster Coordinate System. In 
effect, there seems to be a considerable difference between the original 
notion and hoped-for result of protocols and what seems to be evolving. 

Additionally, in their effort to identify concepts to undergird 
material development, project personnel seem to have either selected 
some which are not as acceptable to teacher educators or, as suggested 
earlier, concept overkill has occurred. Since the concepts which have 
been visualized are not new or uncommon, most likely the latter has 
occurred. Developers may have treated the concepts to be illustrated 
too similarly, assuming that all learners had no advanced organizers, 
preinformation, or experience which would facilitate learning the 
concept. Certainly some concepts are learned more easily than others 
and must be learned differently by different people. 

If a concept- first approach continues to be followed, each concept 
considered for illustration should be plac^^in juxtaposition with the 
very useful *'cone of experience'* to determine whether it is best learned 
and necessarily learned in the context of an elaborate visual experi- 
ence. Films illustrating concepts should be used only vA\en somewhat 
more direct, concrete forms of learning are required. There is little 
reason to develop elaborate and extensive visuals for use in concept 
formation when simpler, less expensive, or less time-consuming 
approaches are available and justifiable. Teacher educators simply 
will not spend more time or money to teach a concept than is necessary. 
All things being equal, given the choice of defining a concept such as 
'*peer group" or having preservice teachers learn the concept by seeing 
an illustrative film or filmstrip, the teacher is likely to use the 
more efficient, parsimonious means. In several instances, in order 
to learn a very few concepts the learner must engage in protocol study 
for a disproportionate amount of time. Consequently, as noted above, 
publishers do not seem to be excited about the possibility of the adoption 
of protocols into the professional curriculum. Of course, cost is 
another deterrent not to be underestimated. 



POINT OF VIEW 

In conclusion, the Smith task force suggested a powerful notion 
which, i f we arc not careful, may become lost. It is possible that 
the sole difference between the way educational theory is learned now 
and the way it would be learned utilizing a protocol is that in the 
latter the concepts are illustrated more elaborately. Elaborate ill- 
ustration may not bo related to the teacher's ability to recall that 
concept when /leccssary in order to better analyze and intei^pret a 
classroom event. I tend to agree that the concept must be learned 
in situ. To me the protocol should illustrate the significant educa- 
tional e\ent rather than the concepts used to interpret it. It seems 
somehow we have gotten the cart before the horse. Once we have recorded 
these events we have, perhaps for the first time, an opportunity empir- 
ically to develop a classroom s i tuat i on- based curr i culum for teachers. 
Concepts, skills, and attitudes will be found to understand and interpret 
those situations. Tlicy will he learned ov discovered in and used in 
relation to these cvcics. These concej^ts can he illustrated as nec- 
essary. It is oni> from events that theor}- can he derived and to them 



that it takes on utility. It seems we may end up with concepts but no 
educationally significant events to which they can be readily applied. 
Are the concepts educationally significant events in themselves? Where 
is the real world of teaching called for in Teachers for the Real World? 
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APPENDIX A 



MASTER COORDINATE SYSTEM 

COMPONENTS OF THE GENERAL PLAN 

In order to develop protocol materials iii an orderly way, it is 
necessary to follow a general plan. The plan should be as comprehensive 
as possible and free of doctrines about what teachers are to be prepared 
to do and how they are to be prepared to do it. If these conditions are 
satisfied, those who are engaged in teacher education would be able to 
take part in a national effort to develop materials without being framed 
by theories and doctrines to which they cannot subscribe. 

The general plan consists of two basic interrelated components or 
sub-plans: one for the pedagogical domain and one for the basic fields 
of knowledge. Each sub-plan will be presented in the form of a coordi- 
nate system comprised of three dimensions. Essentially, protocol mate- 
rials to be developed in the pedagogical domain should be concerned with 
the act of teaching and of learning, with the behavior of teachers, 
learners, and teachers and learners in interaction. In contrast, pro- 
tocol materials in the domain of basic fields of knowledge should be 
concerned with the content of what is taught — or, more specifically, 
with the knowledge about the knowledge that is taught. 

Tlie protocol material? to be developed in either plan should be 
seen initially in terms of the master coordinate system for that plan 
(see Figures 1 and 5). That is, in the pedagogical plan, a decision 
should be made about the levels, behaviors, and settings to be portrayed 
in the protocol materials. In the basic-f ields-of-knowledge plan, deci- 
sions must be made about the levels, types of knowledge about knowledge, 
and areas of knowledge to be portrayed in the protocol material. Follow- 
ing this procedure will help "chart^^ the kinds of protocol materials 
needed as part of a large-scale effort in developing a variety of such 
materials. 



PEDAGOGICAL PLAN 

The pedagogical plan is a coordinate system comprised of three 
generic categories: setting, level, and behavior. These terms will 
be defined later, but for immediate purposes '^setting" refers to the 
context in which behavior occurs, "level" stands for the stage of a 
person's general development, and "behavior" for observable activities. 
Figure 1 depicts the system which is described in detail in the pages 
following. 

In order to indicate the various types of protocol materials that 
can be developed in accordance with the master plan, it is necessary 
to derive subordinate systems. These will be set forth after the master 
plan has been presented. Ttie categories and sub-categories in the 
system below are purposely general. The lack of specificity under the 
category "Teacher- Pupil" or sub-category "Instructional: Skill,** for 
example, allows the developer a high degree of flexibility in deciding 
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Figure 1 

Master Coordinate System of .Settings, Behaviors, and Levels 




upon the specific behaviors he wishes to record. It shbuld be remembered 
that the categories are intended to help assure an adecpaate coverage of 
settings, behaviors, and levels in the protocol materials to be devel- 
oped in all projects taken collectively. They are not intended to be a 
framework for retrieval or for dissemination of materials. 

1. Definitions . 

The major categories are named in Figure 1. We shall first define 
the kinds of settings, and, then the kinds of levels, and finally 
the sorts of behaviors. 

A. Settings . The K:^ntext from which the protocol material is 
taken. 

(1) Classroom . Any room in a school building in which the 
activities carried on are intended to promote learning, 
or more generally, any place where the activities are 
conducted with that intention. 

(2) School , Any establishment for teaching and learning. 

(3) Peer Group . A number of individuals of approximately the 
same age forming a recognizable unit either in school or 
out. 

(4) Family . Any group made up of parents and their children. 

^o^™^ity - The people who live in a district or city 
under the same laws and institutions. 

B. Levels . The periods or phases of a person* s growth from birth 
to adulthood, 

Early Childhood . * The period from infancy to the time the 
child begins school. 

Middle Childhood . The period between early childhood and 
the beginning of adolescence. Roughly, the elementary 
school years. 

Pre- Adolescence . The transition period between middle 
childhood and the adolescent period. Roughly, the junior 
high school years. 

Adolescence . The period immediately preceding adulthood. 
Roughly, the high school and early college years. 

(5) . Adult . Post-adolescent years. 

C. Sources of Behavi, )r. Die observable actions ot' a person, verbaJ 
and non-verbal. 

/ 



(1) 
(2) 

(3) 

(-1) 



(1) ■ Teacher Behavior . Any behavior that a person exhibits as 

he engages in performing the tasks of a teacher, such as 
questioning, explaining, assigning, conferring, and managing 
a classroom or as he takes part in extra-classroan activities. 

(2) Teacher- Pupil Behavior. Any behavior that involves inter- 
act ioi) between a pupil and a teacher. 

(3) Pupil- Behavior . Any behavior that a child or adolescent 
exhibits as he attempts to meet the situations that face 
him from moment -to- moment throughout the day; 

How To Interpret the Master System . 

By identifying each category with code letters, we can indicate 
each three-dimensional cell. In Figure 1, letters CAT identify a 
cell. The first of the three letters (C), stands for the kind of 
setting, the second (A) for the level of . development of the pupil, 
and the third (T) for the source of tht /behavior. The protocol 
materials for this cell would consist or reproductions of teacher 
behavior in classroonis at the high school level. By the same token, 
the protocol materials that depict pupil behavior at the adolescent 
level in a family setting can be indicated by the letters FAP. 

SUBORDINATE SYSTEMS 

HoK the Sub-Cells Were Derived . 

The subordinate systems in Figures 2, 3, and 4 result from expanding 
each of the behavior categories in Figure 1, l^^lile leaving the **set- 
ting categories" and the "levjl categories" unanalyzed. In Figure 
2, category P has been expanded into behaviors that allow the devel- 
opment of pupils. In Figures 3 and 4 respectively, categories Tp 
and T have been expanded into kinds of teacher behavior. Figure 3 
represents a model in which teacher-pupil behavior is included, 
while Figure 4 represents the behavior of the teacher as he interacts 
with peers and others. 



Definitions. 




The definitions of "levels" and "setting** arid thair sub-terms for 
Figurej 2, 3, and 4 are the same as the definitions of these terms 
and sub-terms for Figure 1. 

A. Types of Behavior in Figure 2 . 

(1) Cognitive Behavior . The behavior of pupils that entails 
the acquisition of knowledge: perceiving, conceptualizing, 
inferring, classifying, etc. 

(2) A ffective Behavior . Pupil behavior in the areas of moti- 
vation, valuing, commitment, personal choice, etc. 



(3) Social Behavior . Pupil behavior characterized by such 
processes as social cooperation, ccuipeti tion , authority 
relationships,^ etc. 

(4) S kills . Such cognitive or psycho-motor behaviors as 
spel ling, penmanship, typing, woodworking , gymnastics, 
etc. 

Types of Teacher- Pupil Interaction in Figure 3 . 

(1) Instructional: Cognitive — Interactions in the cognitive 
realm, such as conveying information, building concepts,' 
explaining, diagnosing of difficulty in understanding. 

\ 

(2) Instructional: Affective-- Interactions in the affective 
realm, such as motivating, attitude formation and refor- 
mation, influencing techniques. 

(3) Instructional: Skill — Interactions in the realm of cog- 
nitive skills (such as word analysis , spell in g) and 'phy- 
sical-coordinate skills (such as typing, woodworking, 
gymnastics). 

(4) Classroom Control and Management-- Int er act ions involving 
classroom control and discipline, especially social and 
physical control. 

(5) Personal- Social Development-- Interact ions involvin-g per- . 
sonal development of pupils in such areas as responsibi- 
lity, personal concerns, social relationships. 

Types of Teacher Behavior in Figure 4 . 

(1) Teacher- Civic Groups . The behavior of one or more teachers 
in community groups where educational policies, programs, 
etc., are being considered: parent-teacher groups, open 
sessions of boards of education, special interest groups, 
etc. 

(2) Teacher- Adults . The behavior of a teacher in conference 
with a parent or other adults o/ the community about the 
education and welfare of pupils. 

(3) Teacher- Colleagues ^ teacher behavior in situations in- 
volving other teachers, school psychologists, social 
workers, or counselors where pupil problems, school pro- 
grams, etc., are being considered. 

(4) Teacher- Admini strators . Teacher behavior in conference witl 
principals , supervi sors, department heads , and the 1 ike. 



^ Figure 2 

Expansion of P Category for All Settings and Levels. 




Figure 3 

Expansion of Tp Category for All Settings and Levels 
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Figure 4 

Expansion of T Category for All Settings and Levels 
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3. How to Interpret the Subordinate Systems . 



Using the code letters of each category, we can see by references 
to Figure 2 that protocol materials that depict cognitive behavior 
at the pre-adolescent level in a classroom can be indicated by the 
letters CPaCe. In Figure 3, protocol materials showing concept 
building at the pre- adolescent level can be designated by CPalc. 
In Figure 4, protocol materials that depict teacher behavior in a 
civic group concerned with high school pupils in the community can 
be designated by CoATc. 

It should be noted that there are no empty cells in category P, 
Figure 2, because each type of pupil behavior can be studied in 
every setting for each level of development. Nevertheless, it is 
likely that some types of behavior are more appropriately studied 
in some settings than in others. For exan^le, cognitive behaviors 
can be studied perhaps more appropriately in the school room or 
family than in the broader community setting. 

The Teacher- Pupil category. Figure 3, may have a number of empty 
cells. This is so because teacher-pupil interaction is typically 
limited to classroom and school settings. All the cells bounded 
by peer group, family, and community dimensions are likely to be 
empty. 

Likewise, there are empty cells in category T, Figure 4. For 
example, no protocol materials are likely to be developed in cell 
PgETc because Teacher- Civic group behavior is not ordinarily ex- 
hibited in pupil peer groups. It is easy to identify other empty 
cells in Figure 4 by the same criterion. 

I PLAN FOR THE BASIC FIELDS OF KNOWLEDGE 

Thei term ''basic fields of knowledge*' refers to all subjects of in- 
struction except those in pedagogy. It covers vocational and techno- 
logical kubjects, arts, and the conventional disciplines such as physics, 
history, and mathematics. The plan set forth here provides for the 
development of protocol materials to teach certain things, not ordinarily 
taught irji courses, about the content of these subjects. For example, 
the contetnt of a course in history may be biased against a minority 
group or it may be composed of certain elements of knowledge — concepts, 
values, generali zations--and yet the teacher- in-training may mi^s these 
aspects. The development of protocol materials will help to provide 
instruction in these neglected aspects of the subjects of instruction. 

The Ibasic-skills-of-knowledge plan is a system of coordinates 
made of three generic categories: Areas of Knowledge, Levels, and Types 
of Knowledge about Knowledge. Figure 5 represents this system's co- 
ordinates. The expressions used in Figure 5 are defined below. 

1. Definitions . 

We shall give the uses of the expressions "Areas of Knowledge" and 
"Types of Knowledge about Knowledge" and then define the terms 
designating the sub-categoricS. 



Areas of Knowledge . This expression refers to the various 
classes of arts and sciences. (Over 350 subjects are offered 
in the public schools. To represent these in a three- 
dimensional grid requires that they be reduced to a few cate- 
gories, us is attempted in the following defintions adapted 
from T>iCociner' s Zetetics,) 

(1) Symbol ics . \n area of knowledge that includes language 
arts, mathematics, and logic. 

(2) Arts . The area that includes dramatics, graphic arts, 
music, painting, sculpture, literature, industrial design, 
choreography, and architecture. 

(3) Science of Matter and Energy . The group made up principally 
of physics and chemistry, but also including astronomy, 
geology (earth sciences), and mineralogy, all of which are 
unified by the principle of equivalence of matter and energy. 

(4) Biological Sciences. The group treating living things, as 
exhibited in growth and reproduction including botany, 
zoology, morphology, genetics, and cytology, and related to 
3 above by biophysics and biochemistry, and to psychological 
sciences by physiology. 

(5) Psychological Sciences ^ The sciences of behavior of living 
things including principally individual psychology, group 
psychology, and social psychology. 

(6) Sociological Sciences. The sciences which deal with the 
various facts about, and forms of, group life--demography, 
geography, social institutions and ethnology, related to 
the historical studies by anthropology. 

(7) Devel opment al Sc ienc es . Those disciplines which are con- 
cerned with the past, including cosmic evolution and history 
and pre- hi story of cultures. Among these are cosmogony, 
certain aspects of geology and biology, and the history of 
mankind. 

(8) Sustaining Sciences . The studies aimed at maintaining life 
and advancing its welfare including health, physical educa- 
tion, public hygiene, and, at a more sophisticated level, 
agriculture, medicine, engineering, and technologies (manual 
arts, home economics, metal working, and other vocational 
subjects) . 

(9) Regulative Sciences . The areas of knowledge concerned with 
sustaining cooperation among men as they strive to satisfy 
their individual and collective needs. They attempt to keep 
the various elements of society adjusted to one another. 
Among these sciences are economics, political science, 
jurisprudence, and management (Business Education). 



(10) Disseminat j ve Sciences . Those\ sciences which are concerned 
with the task of transmitting information accumulated from 
generation to generation. Among these are library science, 
pedagogy, and journalism. 

(11) Integrative Sciences. The studies attempting to bring 
knowledge to bear upon such questions as consistency of 
knowledge, man's purpose, and his destiny. Among these 
disciplines are philosophy, theology, and ideologies. 

B. Levels . This term has been defined on p. 19. 

C. Types of Knowledge about Knowledge . 

(1) Kinds of Knowledge . The epistemol ogical elements of in- 
structional content such as laws, concepts, rules, values, 
procedures and facts. 

(2) Logical Operations . Logical relations and such operations 
as defining, explaining, classifying, and valuing. 

(3) Types of Biases . Adherence to a point of view in the 
selection and interpretaton of the content of instruction. 
(In some cases, the point of view may be socially neutral 

as in the so-called new math, or in the different approaches 
to biology, chemistry, and physics. In other cases, it 
may be socially deleterious. For example^ biases about 
races, social classes, labor, management, and the like, can 
be injurious to the well-being of certain groups or to the 
public welfare.) 

(4) Types of Utility. The uses to which the various kinds of 
knowledge and logical operations can be put by pupils and 
adults as they carry on the normal activities of life. 
For example, knowledge may be used in repetitive, asso- 
ciative, applicative, or interpretive ways or in making deci- 
sions or in justifying actions. 

Settings . It should be noted that this category has been replaced 
in the basic fields schema by the category "Ai^eas of Knowledge." 
Thib means that no settings are indicated for the development of 
protocol materials in Figure 5, for "Areas of Knowledge" are not 
settings but realms of content. 

In what settings may materials concerned with the content of in- 
struction be developed? A few settings are suj^gested here as exam- 
ples. Some may te taken from the classroom. Others from public 
school textbooks, and still others from newspapers, magazines, and 
radio and television broadcasts. For example, protocol materials, 
to show the utility of certain elements of scientific knowledge 
may be taken from articles in newspapers where such knowledge is 
being used. The analysis of the protocols would attempt to show 
how the knowledge is used in an article as well as how the reader 
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uses his own knowledge as he reads it. Similarly, protocol mate- 
rials to depict elements of knowledge — concepts, values, general- 
izations, etc. --in history may be taken from classroom discourse 
where the content of iiistory is being discussed. The analysis of 
the protocols would help the teacher- in-training to identify the 
elements as they appear in teacher- pupil interaction. 

3. How To Interpret the Basic- Fields-of- Knowledge Plan . Protocol mate- 
rials developed to show how economic knowledge studied in high 
school is used in out-of- school life are placed (Figure 5) in cell 
RsAU. Similarly, materials that are to be used to show the racial 
biases of classroom discourse in a junior high school course in 
history are placed in cell DsPaB. If protocol materials are devel- 
oped to show the elements of knowledge in elementary school geog- 
raphy, they would be located in cell SoQiK. Other materials can 
be distributed in their proper cell by reference to the various 
categories in Figure 5. 

ILLUSTRATIONS OF THE USE OF THE MASTER COORDINATE SYSTEMS 

It might be helpful to illustrate the use of each of the two master 
coordinate systems, that in the pedagogical domain and that in the basic- 
f ields-of-knowl edge domain, by citing the development of two fictitious 
protocol projects, one for each of the domains above. 

1. A Proposed Project in the Pedagogical Plan . A developer decides in 
terms of his own criteria, that protocol materid is needed to pro- 
vide experience in the interpretation of group relationships in the 
classroom. He must first decide upon the precise group concepts 
that he wishes tc have reflected in his protocol material. He next 
defines the concepts of reference group, group norm, and group rein- 
forcement. He must then decide upon the level at which his protocol 
material will be made. Since group relationships are so significant 
at the adolescent level, he decides to develop material portraying 
students in interaction at the high school level. Thus, in Figure 

1, he has chosen to work at the adolescent level (A). He has, of 
course, already decided upon the source of behavior in Figure 1 — 
pupil behavior (P) since it involves relationships among adolescent 
students. He next must decide upon the setting for this interaction. 
He vacillates between showing this interaction in the peer group 
outside of school or in the classroom. Because he is anxious to 
have this interaction directly related to teaching, he decides upon 
the classroom as a setting. He has chosen the classroom category 

(C) in Figure 1. 

He next must decide more specifically on the kind of behavior that 
he wants to occur. Since he is dealing with pupil behavior. Figure 

2, in which pupil behavior is further specified, is most pertinent 
for him.* In considering the possible behaviors, he decides that 



*Figures 5 and 4 are not pertinent because they deal with refinements 
of teacher behavior and of teaclier-pupi 1 interaction. 
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both the affective (AF) and social (SI) aspects of behavior are 
likely to be involved in the type of group interactions he intends 
to portray. These, then, aro the types of behaviors which will be 
significant in his protocol material. 

In this process of planning by the coordinate system, the developer 
of protocol materials has become highly specific about the concepts, 
the levels, the sett ings,*'and the types of behavior that his mate- 
rial will portray. In turn, such specification will aid in the 
national effort to develop a variety of protocol materials without 
extensive overlapping and repetition. 

2. A Proposed Project in the Basic- Fields-of- Knowledge Plan . A pro- 
fessor of education becomes concerned about the kinds of biases 
. (racial, social class, etc.) that are often reinforced in the 
public schools. He particularly wants his teachers- in-training 
to become aware of the frequency with which teachers unintentionally 
convey biases to students in their classes. Referring to Figure 5, 
he may first decide upon the area of knowle^^ge in which such biases 
are likely to be conveyed. The sociological area would be an ob- 
vious choice, 'but the systematic presentation of certain biases 
through the selection and discussion of literary material interests 
him more. Thus, he has decided on the symbolic category (Sy) as 
the area of knowledge the teaching of which will be portrayed in 
his protocol material. He next decides that he would like to show 
this at the elementary level when many biases are being formed. 
Thus, he selects the level of middle childhood (Ch). He has, of 
course, already determined that types of biases (B) will be the 
knowledge about knowledge that will be portrayed in his protocol 
materials. 

Once again, the developer of materials has necessarily and desirably 
become highly specific about the dimensions that will be portrayed 
in his protocol material. Once again, this will help future devel- 
opers to determine what areas in the basic-f ields-of-knowledge plan 
still have not been sufficiently developed in the' form of protocol 
materials. 



Since the cells that include teacher- pupil interaction in the class- 
room at all levels are primarily those in which most teaching problems 
fall, these cells shall be given top priority for the next year (1971- 
72). The second priority should be given to any of the remaining cells 
in the pedagogical plan, but with' the provision that not more than 20 
percent of the new projects can be assigned to the second level. 



PRIORITIES IN THE PEDAGOGICAL PLAN 



Priority 
Level 



Setting 
Classroom (C) 



Behavior 



Level 



2 



Peer- group (Pg) 
Family (F) 



Teacher- pupil (Tp) 
Pupil (P) 



All levels 



Any (E,Ch,Pa,A,Adt) 
Any (E,Ch,Pa,A,Adt) 



2 



Pupil (P) 



Priority 
Level 



Setting 
School (S) 
Community (Co) 



Behavior 



Level 



.2 



Teacher (T) 
Teacher (T) 



Any (E,Ch,Pa,A,Adt) 
Any (E,Ch,Pa,A,Adt) 



2 



PRIORITIES IN THE PLAN FOR BASIC FIELDS OF KNOWLEDGE 

In the first phase of this program to develop protocol materials 
on a nationwide scale, it appears that some arbitrarily determined dis- 
tribution for the two plans would be appropriate. Accordingly, it is pro- 
posed that twenty-five percent of the funded projects be allocated to 
the basic-fields-of- knowledge plan, and that proposals in any cell of 
the plan for the basic fields be considered. 
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INTRODUCTION 



The Romans case grain for birds to scratch to guide their moves in 
public affairs or to predict outc-omes of an undertaking. The patterns 
of grain left by the^birds were interpreted as. good or bad omens. Had 
we followed that old Roman custom, the hen-scratchings might have 
augured three years of very interesting, challenging, and often frus- 
trating work in the development of multi-media protocol materials. 

The justification for writing a case study of the Univp-rsity of 
Colorado Protocol Project is not that the work was interesting or that 
we finally produced something with 3 years of federal money. What 
warrants reporting are the ways in which the project staff came to 
understand a new and sophisticated idea in teacher education and the 
expe::iences of the staff in developing, testing, and disseminating 
protocol materials. We will share the insights, the methods, the prob- 
lems, mistakes, and successes of the University of Colorado Protocol 
F^roject with readers who might wish to develop similar materials and 
with those who are curious about how an educational idea gets translated 
into an instructional product* 

Sumc brief mention of who we are and what resources we had to work 
with may be helpful since the backgrounds and experience of the staff 
and the amount of the U* S. Office of Education grant were to influence 
decisions during the life of the project. Hie academic interests of 
the staff .suggested the general conceptual framework within which we 
worked and the specific concepts we chose to develop. Our inexperience 
with media techniques and technicians accounted partly for the many 
fumbles with videotape, 35mm slides, and 16mm color sound film. 

The total grant for the 42 months, July 1, 1970 through December 
1973, was about $165,000, averaging just under $4,000 a month* No more 
than two persons ever worked full-time at once on the project, although 
part-time clerical and professional assistance came from several persons 
during the 3 1/2 years. In the long run, a relatively low budget,. some 
poor luck with contract technicians, and a resolve to complete the mate- 
rials and get them out turned the Colorado project into a "do-it-your- 
self' operation with the project staff doing much of the technical work 
and all of ^he dissemination. 

The director of the project, Celeste Woodley, and the ?,ssistant 
directors, Ina Mullis (1970-72) and Laura Driscoll (1972-7 3) were 
responsible for the development, testing, and dissemination of the 
University of Colorado protocol materials. Woodley is a member' of the 
faculty of the School of Education; Mullis is currently in the Research 
and Analysis Division of National Assessment of Educ^f^onal Progress, 
Denver; and Driscoll is a doctoral student in Educational Research and 
Evaluation at the University of Colorado. The academic backgrounds of 
the three are in social science education or research and evaluation; 
their working experiences are in teaching, the <lnalysis of teaching, 
curriculum development, product and program .evaluation, and adminis- 
tration of teacher education programs. When the project began, the 
staff's direct experience with media was limited to classroom video- 
taping and threading old movie projectors. 
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The university agency which holds the USOE grant is the Center 
for Education in the Social Sciences, University of Colorado, an orga- 
nization interested in improving education at all levels, particularly 
spcial science education, by strengthening the ties and the dialogue 
between the School of Education and academic departments in the College 
of Arts and Sciences. The background, experience, and organizational 
affiliations of the staff contributed to the frame of reference and to 
the content of Colorado's protocol materials and to the style and for- 
mat of the final products. 

BEGINNINGS 

To begin, we had to understand protocol materials. From there we 
chose our concepts and finally learned about technical quality the hard 
way. 

Scrambling for a Referent 

The USOE request for propo?^^ils (RFP) that came to the Center in 
April 1970 contained ideas and phrases that were not common in our 
repertoire: protocol, behavioral specimens, segments of behavior, ■ 
protocol materials, autlientic representations of educationally rele- 
vant behaviors, the distinction betv/een protocol materials and training 
materials. Uncertainty about the nature of the materials described in 
the RFP caused us to read many times those few pages of description and 
definition. We studied Teachers for the Real Wc^id in which B. Othanel 
Smith had described protocol materials and set t^em in the context of a 
much larger instructional scheme, and we spent a lot of time discussing 
and refining our own interpretations of the nature and meaning of pro- 
tocol materials. The initial struggle to understand the concept 
^'protocol materials" gave us an appreciation of the task of concept 
analysis that was to become central in product development. To begin 
to understand protocol materials for ourselves, we needed a definition 
or rough idea of the concept. We profited from knowing what protocol 
materials were not; more examples would have been helpful, and delin- 
eation of specific characteristics or attributes might have provided 
good clues as to the nature r>f protocol materials. By the time we 
began to develop materials in the summer of 1970 we knew that protocol 
materials linked concepts with their behavioral referenf^ as repre- 
sented in filmed or video vignettes. It was clear that the two critical 
aspects of development were concept selection and analysis and Ihe re- 
production and representation of descriptive behaviors in an appropriate 
media. 

Conceptual Dimension . From the several areas suggested for proto- 
col development, we chose to'work with concepts relevant to teacher- 
pupil classroom interaction. Initially we had proposed to illustrate 
concepts such as "motivation," "involvement," "ethnocentric ity,". "cog- 
nitive style," "value formation," "classroom climate^" "concept attain- 
ment," "hypothesizing," ''resolution of conflict," and "evaluation." 
These concepts were chosen because they were referred to frequently in 
teacher education texts. This smorgasbord seemed plausible at the time, 
but by the fall of 1970 it was difficult to justify the shot-gun 
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approach. It was also apparent that we vvould have to drop plans to 
use, in final product, vignettes from our large collection of 1/2- inch 
video tapes of classroom teaching. Earlier it had seemed, reasonable 
that behaviors ex«nplifying all of the concepts we chose to portray 
would occur somewhere in the 60 hours of video footage. That idea 
was not inconsistent with the notion of protocols as "slices of life" 
thac was pervasive early in the program. It seemed practical to select 
those slices from existing resources. We had begun to catalogue the 
tapes by segments according to conceptual content, noting quality of 
sound and cpaality of picture. The next step, we thought, would be 
editing the segments into a flow of examples emphasizing the behavioral 
attributes of, a particular concept, adding some titles and music, con- 
verting to kinescope, writing a guide, and Voila! protocol macerials. 
It was not quite that simple. Figuring out just what we v;ere supposed 
to be doing and how we were to do it took a while. 

The quarterly meetings of the protocol directors and members of 
the Leadership Training 'Ins ti tute (LTI) provided the feedback necessary 
for us to find out, if somevJiat indirectly, how we were doing with the 
two major criteria of protocol materials production, i.e., the con- 
ceptual validity and technical quality. These "show and tell" sessions 
provided important verbal and non-verbal clues. A viewer's question-- 
"What is the concept?" — immediately following p. presentation, was a 
pretty good indication that conceptual quality was doubtful. After 
three or four wrong guesses from others in the audience, one might 
prudently consider beginning all over again. 

T echnical Dimensi on . The real meaning of technical quality grad- 
ually emerged from the reactions of the directors and LTI audience to 
jumpy cameras, sterotyped acting, blurred visuals, and sli^rred sound. 
All these flaws were tolerable when the products were viewed as rough 
draft films or video tapes, but whatever delusions we ha^ about pre- 
serving the home-made look in a final product were laughed away. The 
criterion ''Commercial Broadcast Quality" became the word of the day by 
late 1970, and the implications of what that meant in /terms of time, 
money* and precision also prompted a reassessment of our production ( 
plans. 

Steps in Development . The trial and error process- of the early 
months of the project indicated the following steps as basic in proto- 
col material development: / 

^\ 

1. Developing a conceptual framework that has utility for understanding 
the teach ing- learning process; 

2. Selecting from the framework and further defining concepts for 
protocol development; ' % 

3. Identifying the properties and behavioral attributes of the con- 
cepts; 

4. Deciding on the appropriate media; 

5. Loosely scripting classroom episodes exemplifying^ both instances 
and non-instances of the concepts; 

6. Filming or taping the vignettes; 

7. Editing the footage; ' ' 

8. Adding appropriate titles, credits, music definitions, questions, 
and models; 

0. Writing supplementary teacher's and student's guides. 



REASSESSMENT: STRATEGIES AND TACTICS OF SURVIVAL 



The skirmishes of 1970 gave us a better look at our target and 
suggested some strategies for attack. ^ The target, it turned out, was 
a formidable task, more complex and sophisticated than we had realized. 
Precise^ exemplification of well-dofined concepts in various media forms 
of high technical quality was required. 

The Conceptual Frair.ework and Selection of Concepts 

The selection of concepts was the first concern in our assessment. 
We wanted to stay with teacher-pupil interaction in the classroom 
because we' w6re comfortable in that area and we felt we could success- 
fully portray concept-behavior linkages there. It w^s also an area of 
high priority for the entire protocol program. We were convinced, 
however, that producing protocols for concepts like "ques tioningf' or 
"punishment and reward" or "probing" did not get at what is basically 
and crucially missing in a teacher's ability to interpret classroom 
interaction. The rejection of specific, skill-related concepts was not 
a denial of the significance or validity of such concepts, just a dis- 
agreement with the level and scope of conceptual focus. Furthermore, 
other projects were dealing successfully with many of those concepts. 
There was no need to be redundant. 

We were interested in equipping teachers with the conceptual and 
analytic tools to change their own behaviors. It seemed to us that 
teachers first needed an understanding of a theoretical fra^iework that 
made sense of the instructional process itself. Then the meaning of 
ary particular specific concept and the interpretation of associated 
behaviors could be placed in a theoretical, as well as in a situational, 
context. There would be much greater opportunity for teachers to com- 
prehend what was happening in teacher- pupil interaction and to go beyond 
the specific behaviors to some idea of why they were happening. Refer- 
ring to a conceptual model, teachers could modify their decisions at 
the point where most impact on consequent interactional behaviors might 
be made. 

We derived two models to guide our thinking and selection of con- 
cepts. One, derived primarily from the work of Staked and G ,2 is 
a decision-making model of the process of instruction (Figur ) which 
indicates major conceptual components o£ a theory of instructional 
decision making and suggests the sequential and functional interrela- 
tionships of these components. The concepts we chose to exemplify are 
related to one another functionally and sequentially in this conceptual 
model as generally indicated by the arrows between components. We had 
wished to analyze and illustrate at least one concept identifiable in 
each of the major components, but we had neither the time, the money, 
nor the talent for so large an undertaking. We chose to illustrate six 
concepts, one related to the entire process of instruction, one to 
antecedent conditions, one to selection of instructional alternatives, 
and three to interaction. CFwo were later combined in one unit.) 
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Before we selected specific concepts and defined tho^'' concepts, 
we had to resolve the question of the level of specificity of concepts 
with which we would work. Concepts can be arranged in a hierarchy of 
complexity depending upon the number of discrete attributes that char- 
acterize them and the logical linkages and dependencies among the 
attributes. There are large and complex concepts, middle-level concepts, 
and specific and simple concepts pertinent to instructional theory. At 
what level in the conceptual hierarchy was it useful and practical for 
us to work? Our frame of reference dictated a focus on middle-level 
concepts that provided, we believed, context for understanding and in- 
terpreting the more specific behaviors that ^characterize more narrowly 
defined concepts. We were'not unaware of the risk of ambiguity and the 
difficulty of representing specific behavioral attributes when we 
decided to portray these more general middle-level concepts. The trun- 
cated outline in Figure 2 indicates, in relation to some other concept- 
ual categories, the concepts portrayed in the set of products developed 
in 1971 and 1972 by the Colorado project. The specific titles of the 
five products and the concepts exemplified follow. 

Title Concept 

1. Conceptual izirg the Process of Instruction 

2. Learners and Their Characteristics: Impli- 
cations for Instructio :al Decision-Making 



3. Verbal Interaction in the Cognitive 
D^jnension 

4. Fair Verbal Behavior 

5. Organizing Facts To Teach Meaningful 
Relationships 

The middle-level designation of these concepts is relative, e.g., "Pro- 
cess of Instruction'' is a less general, less complex concept than ''Edu- 
cation'' and is subsumed under it. "Fair Verbal Behavior" is more 
general, more complex than "Initiating and Reacting Behaviors" and en- 
compasses them and the even more specific "Accepting," "Encouraging," 
and "Task- Oriented Behaviors," for example. 

Linking Concepts to Observable Behaviors 

We discovered that middle-level concepts become quite a mouthful 
when verbalized--"process of instruction," "attitudes toward school," 
"verbal interaction in the cognitive dimension," "fair verbal behavior," 
"concept teaching." They are also unwieldy and hard to pin down to 
specific, unambiguous indicators. 

The process of analyzing concepts, defining them, and listing 
their behavioral attributes was the most difficult and critical task 
of protocol materials development. It is the essential step between 
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NOTE: The heavy outlines indicate categories in which protocol mate- 
rials were developed by the Colorado Project. 



selection of the concepts and the technical work with media. Analysis 
o£ the selected concepts must yield definitions and characterizing 
behavioral attributes. (See "A Catalogue of Concepts in the Pedagogical 
Domain of Teacher Education/ p. 54.) It is the behavioral attributes 
or indicators of the concept that are observable and able to be cap- 
tured on film or tape. We defined "Fair Verbal Behavior" for example, 
as teacher verbal behavior that follows a consistent standard toward 
all students and is free from bias, dishonesty, and injustice. That 
definition was derived from student descriptions of teachers and class- 
room experiences . available in general and research literature on class- 
room teaching. Students have a concept of fairness, and how a teacher 
measures up as fair or unfair seems to affect student behaviors. To 
treat "Fair Verbal Behavior" as an aspect of classroom interaction and 
as a significant concept seemed justified. The behavioral indicators 
of "Fair Verbal Behavior" were identified from the definition and from 
specific student perceptions. A fair teacher: 

1. Praises students' ideas rather than students personally. 

2. Praises students when they deserve it and corrects then when they 
make a mistake or misbehave. 

3. Explains the mistake or misbehavior rather than criticizes the 
stuuent personally. 

4. Asks students to clarify statements or defend actions in order to 
determine whether to praise or correct students. 

5. Is not capricious and arbitrary in dispensing praise or correcting 
students. 

It was helpful to us to list also negative instances of "Fair Verbal 
Behavior," e.g., praises students who don't deserve it. This analytical 
aspect of development required a heavy investment of time. 

Once the obser»/able behavioral characteristics of any concept were 
delineated, we had to decide on the appropriate vehicle for media repre- 
sentation. Our deliberations were not terribly complex. They were 
based in some cases on LTI advice and sometimes on experience, The 
thinking went something like this: If continous flow of dialogue and 
physical movement arc both necessary to identify the behavioral indi- 
cators of the concept, use video tape or film. If non-verbal expression 
or symbolic records convey the attributes of a concept, use slides (or 
filmstrip). If the subjects' verbal behaviors add a necessary dimen- 
sion, use cassette or audio tape, but add slides for interest or 
clarity. If the behaviors are difficult to distinguish initially with- 
out some help, use overprint, arrows, or other focusing aids. Since 1/2- 
iiich video converted to kinescope looks like reruns of "Our Gang" com- 
edies, use 2- inch tape for video. , Since good sound is critical, use a 
studio set-up and plenty of microphones when filming. Since there are 
difficulties with the compatibility of the various video formats, 16mm 
film is preferred. Since color film is usually more attractive to the 
viewer, use color. Since teacher's guides are often a drag, keep them 



brief and put instructions on the film where feasible. What actually 
happened on the technical end of protocol materials production at 
Colorado is another story. 

The Root of All Evil: Technical Incompetence 

All great dreams have sub-themes tha:: run under the main plot. In 
our case, the sub-plot almost overwhelmed the serious theme of producing 
conceptually sound protocol materials. The struggle in the wings with a 
varied assortment of incompetent media specialists almost closed the 
show. The judgmental errors in selecting these persons were ours, and 
we have vowed to give a lie detector test in the future to every appli- 
cant who claims to know anything about media. Some highlights from our 
experiences may forewarn the reader who contemplates . a multi-media 
venture. 



Where Is the Picture? A Video Vignette 

The VTR is on and reels are turning; camera is on; two 
plugs are in the wall. Students and teacher are engaging in 
behaviors perfect for the protocol. 

"Everything set? Are you getting a good picture?" 

"Don't worry about a thing." (Peers into camera.) 
"Picture is just great!" 

Twenty minutes go by. Kids can hardly wait to see them- 
selves on the monitor. Rewind. Switch to play. » 

"Where's the picture? IVhere's the picture?" 

"Ooops. Forgot to plug the camera to the VTR." 

A Study in Gray and Gray 

"What this kine needs is some sharp title cards and music 
under the introduction. I'll- pick oift the mood music for you, 
I know the beat to look for. You just take your titles over 
to Nlurph; he'll shoot them with his Boley." 

Murph is stuck away in' the basement of the oldest building 
in town. The "Boley" has no tripod, but "My hand is steady." 
The poor light and the steadily shaking hand produced a piece 
of film that the processor tried to expose twice. 

But the viewer hardly notices it go by at all; he is in 
shock from the variation of "Babes in Toyland" that "sets the 
mood" for "Conceptualizing the Process of Instruction." 

The Sound of Speed and All That 

"Glad you decided to go to 16 mill. Queen of the media 
. . . I' 11 get on the phone to my contacts in the morning and 
put in the order for everything we need . . . Ship it by Air 
Cargo the next day . . . Let's see: positive-negative, double 
reversal, optical sound and a double-sync projector, edge 
markings and the Movieola, crystal sync recorder and three 
Nagras, Bolex, Eclair, and Aeroflex, booms and wireless ..." 
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"Wait a minute. You just lost us." 

"You've got to remember. With me as producer-director 
you move with the sound of speed." 

("Sound of speed? Is that — CRASH?") 

Lest you be afraid to buy a roll of film from now on, be advised 
that we were trying to keep our expenses down and had not taken our 
business to well-established film companies. Their prices were in the 
range of $1,000 a minute for 16ram color film and $20.00 an hour for 
35mm work. Much later we did find some very able professional people 
to help us whose charges fpr high quality work were minimal. 

"Do- It- Yourself: The Last Resort 

There were many times in the last 3 years that the three of us 
felt we had embarked on new careers as graphic artists, paste-up 
experts, script writers, film editors, sound women, and recording 
artists. We made almost all of the title and credit ^cards for copy 
stand photography and for revision of black and white films. We 
used press-on letters, 9x12 negatives and colored gel or acrylic, en- 
larged color prints, and poster paints. The selection and sequencing 
of slides for filmstrips, the scripting of the dialogue for t*he film- 
strips, the selection of music and special effects, the locating of 
subjects to be in the films or filmstrips were all part of the job. 
The sessions with the Movieola, sync' in g lips with sound, lasted far 
into many nights. We had 6,000 feet of color film and 8,000 feet of 
sound tape that we had to see and hear together in order to select 
the most illustrative instances for two of the products. Once we had 
identified the segments, we searched for and found a competent film 
editor who didn't mind us working right with him until we were satisfied 
that we had what we wanted. 

"Do-it-yourself" is not quite right. There were mary persons who 
donated time and talent to help us out. Cooperation and interest were 
demonstrated by students, teachers, school administrators, professionals 
and employees of film sound studios, amateur actors in a local theatre 
group, and many persons whom we talked with but never saw. ^ The gener- 
osity of the Boulder Valley District Re-2 and Adams County District 
#50 schools in allowing us to use their classroom facilities for filming 
made much of the work possible. With all of this help, our color films 
cost not more than $500 per minute instead of the usually quoted price 
of $1,000 per minute. 

TESTING, EVALUATION, REVISION 

Testing, evaluation, and revision of the protocol materials were 
ongoing processes that began as early as the fall of 1970. 

Pilot Testing 



Pilot tests provided information on student and instructor percep- 
tions of technical qual' ty and on the clarity and significance of concepts. 



These early trials were instrumental in revising the films and in 
writing the instructor guides and student materials. 

In late 1970, the first film xinit, "Conceptualizing tlie Process 
of Instruction," was piloted in education classes at the University of 
Colorado, Colorado State University, and at sites in Florida. The 200 
students who used the materials at that time reacted positively to the 
idea of protocols, hut were negative about the quality of the film and 
the structured presentations. "You're forcing us to see something in 
this that we just can't (or don't) see" was a typical comment. This 
kind of student reaction was given much weight in revising' the film and 
instructor's guide. 

The revised unit on the "Process of Instruction" was again pilot 
tested under similar circumstances in the spring of 1971 along with 
two new protocol materials units, "Verbal Interaction in the Cognitive 
Dimension" and Part I of "Learners and Their Characteristics," a film- 
strip on attitudes toward school. We learned that instructors did not 
always understand the suggested uses and often had different ideas about 
the most advantageous way to present the materials. Their comments and 
reactions led us to write a more detailed rationale for the units, more 
explicit directions, and suggest alternative teaching procedures for 
each unit. All five products were readied for the major field test in 
this way. 

Field Testing 

In the spring of 1972 our five multi-media protocol products were 
field tested at 15 colleges and universities around the country. 

The Subjects . Five social science methods classes served as a 
control group; the ten experimental classes consisted of five general 
methods and five social science methods classes. The 15 different 
-test sites represented a diversity of urban and rural regions. The 
instructors were selected from among willing persons suggested by 
friends around the country. 

T he Instruments . The evaluation instruments were.^constn.\cted by 
Ina Mullis, with assistance from Gene V Glass of the Laboratory of 
Educational Research at the University of Colorado. The instruments 
were drawn up according to the objectives of the project and in order 
to answer two evaluative questions: "How effective did the participants 
judge each product to be?" and "Is the protocol materials approach more 
interesting then conventional approaches in teacher education^courses?" 
The questionnaires included a pre- and post-test, i/f^ended to measure 
concept acquisition; instructor and student background information 
forms, which yielded such data as years of teaching experience, degrees 
obtained i*^ specific areas, age, sex, number of hours taken in differ- 
ent subject areas, previous non-teaching experiences, and future plans; 
separate criterion tests for each product; and instructor and student * ^ 
evaluation forms for each of the five products. The purpose of the 
criterion tests was to assess whether or not the objectives of each 
product had been met. They differed in form from one product to 
another, depending upon the nature of the activities prescribed for 
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each unit. The evaluation questionnaires v^re similar for all five 
products. The 15 Likert scale items on each form were intended to 
measure the interest and perception of the significance, clarity, suf- 
ficiency, and effectiveness of each unit. 

r 

The Instructions . All of the field test materials and instructions 
were air expressed to the instructors during the last week in March 
1972. Complete instructions and an inventory of contents of the testing 
box were included with the shipment. For every instrument administered 
a large, stamped return envelope was provided for immediate return of 
questionnaires or tests. Instructors in the experimental classes had 
the prerogative to decide when and in what order to teach the protocol 
units. 

Problems . The field testing procedure could have been improved. 
We were fortunate that we found instructors to* participate at all con- 
sidering the late date. We should have sent the materials a month 
earlier than we did. Some instructors did not have enough time to com- 
plete all the units before the end of the semester. Confusion about 
the nature of protocol materials might hjg:ye been avoided had we con- 
ducted an orientation session for the instructors before they used the 
materials. 

. Evaluation 

Returns of the data were disappointing. Two instructors returned 
the box of materials unopened. Only two instructors returned all of • 
the data; there were partial returns from the others. More than half 
of the students did not take all the criterion tests, mostly because 
the tests were too long and too difficult. Incomplete returns prevented 
us from analyzing the relationships between the background information 
data and achievement or attitude scores. We were able to compute an 
analysis of co-variance on post-test scores using pre-test scores as 
a covariate for experimental and control groups and to obtain mean 
scores for criterion tests for the two experimental groups and mean 
scores for instructor and student groups by product. 

The staff worked with an outside evaluator. Mar/ Lee Shiith of 
the University of Colorado, on the scoring and tabulation of the data. 
Smith analyzed the data and prepared the evaluation report. 

The results of the field testing can best be summarized by examinitig 
the responses, generally speaking, to the two primary evaluative ques- 
tions viiich were embodied in the instruments. In answer to '*How 
effective did the participants judge each product to be?" it can be 
said that both the concepts and the activities used in the units were 
considered worthwhile and significant by both students and instructors 
for all five products. The interest level of all units but one vras 
rated as "more*' or "much more" interesting than what "normally takes 
place in my education classes." Two units suffered somewhat from a 
lack of clarity, according to the respondents. 



In order to measure the difference between the protocol approach 
and conventional approaches to teacher education, the responses of the 
two experimental groups and the responses of the control group to 
the post-cest were statistically compared. The questionnaire was 
divided into five sub-sets (groups of three or four items) which related 
to different instructional objectives. Also used to judge the differ- 
ence were some of the items on the evaluation cpaestionnaires . Both 
students and instructors apparently perceived that "Verbal Interaction 
in the Cognitive Dimension^' and ''Fair Verbal Behavior" taught the 
students more than they would have ordinarily learned in a teacher 
education course. Instructors thought that "Learners and Their Char- 
acteristics" offered a better approach than the traditional, but 
students did not. Students thought that "Organizing Facts To Teach 
Meaningful Relationships" (on concept teaching) was an improvement on 
conventional approaches, but instructors thought it was no better. 

Judgments 

The responses were studied by the evaluator, the project director, 
and a social studies methods instructor. Each wrote an independent 
judgment of the implications of the data for revising, disseminating, 
and using the five products. The judgments reflected agreement that 
revisions had to be made, but also indicated that the materials were 
essentially useful and should be disseminated as widely as possible. 

Revision 

Final revision of the materials according to the directions in- 
ferred from the data was undertaken in the fall of 1972. All black 
and white films, for example, were converted to color effect by tinting 
the kinescope in printing and by adding new color titles. Some of the 
episodes were shortened, and some clarifying questions or instructions 
were added. All of the instructor's' guides and student handouts were 
re- written and shortened. The main emphasis there was on revising the 
criterion tests, included in the guide to make them less time consuming, 
easier to understand and use, and more uniform. 

The field test results shewed the materials to be generally worth- 
while and interesting. Better products had resulted from revision, and/ 
there was no reason not to disseminate the materials as quickly and as 
inexpensively as possible. 

DISSEMINATION 

It was a great feeling to survey a shelf full of finished mate- 
rials that were apparently useful and wanted. By the spring of 1973, 
the materials were ready for a publisher and distributor, and contacts 
were made with the publishing industry. The responses of the commercial 
publishers were less enthusiastic than anticipated and probably for 
good reason. Although publishers responded favorably to the content 
and technical quality of the materials, they had concerns and questions 
about the size of the market, cost and time problems of reproducing 
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and marketing multi -media materials, the packaging and warehousing 
problems, and the high mark-up price which would pro|iibit many potential 
customers from purchasing the materials. The prospect of any comnercial 
publisher marketing the products looked extremely blea)^. Although the 
market was small by commercial standards, materials like these were 
much in demand. To spend more time trying to find a commercial pub- 
lisher seemed ill-advised. We wanted to get the protocols out wliile 
they were still relevant and up-to-date. This looked like a time, for 
the "try-it-yourself ' strategy. V 

Authorization to secure final copyright and to make the University 
of Colorado the publisher and disseminator of the materials was applied 
for from USOE. Such authorization was granted for a 5-year period, with 
permission to sell the materials on a non-profit basis. 

It Pays To Advertise . 

An advertising campaign was launched on May 1, 1973 by sending out 
10,000 one-page brochures announcing the availability and cost of the 
materials. These wer^ sent to persons on the mailing lists of the 
American Association of Colleges for "Teacher Education, the fcRIC Clear- 
inghouse on Social Studies/ Social Science Education , and the Social 
Science Education Consortium. The cost of printing, mailing lists, and 
mailing was about $650. 

In anticipation of requests for preview, rental, and .sale, we 
ordered 25 prints made of each film and filmstrip. The total expense 
for the prints and the appropriate sets of written materials came to 
about $5,500. The selling , price of each protocol materials uuit was 
set to cover expenses, including mailing and packaging costs, and to 
provide for replacement of the unit. 

The response from advertising has been very gratifying. Requests 
have come from agencies at all levels of education. As of the end of 
December 1973, there had been about 5100 mail and telephone contacts. 
Therf had. be e^ over 100 requests for more information; 200 requests 
for previews; 50 rental orders (at $5.00 per unit); and 75 sale orders. • 
The toal dollar value of orders by the end of December was just over 
$7,000. 

Producticji Goes On . . . 



With the dissemination problem temporarily under control, the 
staff turned attention back to development of protocol materials in a 
new area in the third year of the grant from USOE. 

In the fall of 1972, while still evaluating and revising the 
earlier products, we began to plan production of four filmstrips deal- 
ing with social science concepts. The goal for 1973 was to select and 
portray ('concepts significant for developing an understanding of social 
behaviot as it related to the phenomena o^f social stability and social 
change. TKe search for a theoretical and integrating framework within 
which to locate and explore concepts related to social conduct directed 
us to role theory as the general area in which to work. The role 



orientation appealed because it offers an interdisciplinary approach 
to the several social sciences. The concepts selected for protocol 
development are *'role conflict," "conformity," "consensus," and 
"position, identity, status, and role." These' concepts are useful in 
exploring the "fit" between individual behavior and group behavior, 
and they have important implications for examining the nature of social 
change or social stability. 

At this writing "Role Conflict" and "Conformity*' have been completed, 
^^^le Conflict" is a filmstrip and cassette unit, using six vignettes to 
illustrate expectations, positions, and behaviors knd their relationships 
in the dj^lemma of conflict. "Conformity" is also presented on filmstrip 
and"x:a:^ette. The concept is treated according to its various defini- 
tions, each of which is illustrated by one or more episodes in the 
filmstrip. There are no instructor's guides for the materials; direc- 
tions and definitions are part of the filmstrip. Research has bieen 
completed for "Consensus" and the script is in preparation.. The last 
unit on "Position, Identity, Status, and Role" will. be prepared in 
February. Field' testing on all four units will be carried out in the 
spring of 1974. In the preparation of the four units on concepts from 
role theory, the Colorado project has been fortunate to have had as 
con<^ultants Bruce J. Biddle and Barbara J. Bank of the Center for 
Research in Social Behavior, University of Missouri. 
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INTRODUCTION 



If protocol materials are to make a programmatic contribution to 
the improvement of teacher education on a nationwide basis, certain 
objectives must be realized. Other papers in this publication deal 
with some of those objectives; for example, technical critieria for pro 
duction of materials must be specified and demanded. Individual 
protocols must teach concepts. The pedagogical and technical knowl- 
edge necessary to achieve criteria associated with both of these 
general goals has been growing during the past 4 years. Protocol 
materials which are being developed at the present time appear to 
be far superior on both these dimensions than were the prototypes of 
1970. 

Despite these real and sizeable advances in the program, however, 
much must be accomplished to reach the original goals concerning, 
national impact on teacher education programs. From the earliest 
planning stages of the protocol program, the need for conceptual 
mapping of the pedagogical domain of teacher education was recognized. 
Such a map would presumably aid in making decisions about viiich con- 
cepts, or at least which groups of concepts, should be developed into 
protocols. Through that mapping effort and, it is hoped, through 
mother similar efforts still to be planned, protocol materials devel- 
opers would receive guidance about undeveloped concepts in the 
pedagogical domain. From the outset, this conceptual mapping was 
conceived as a catalogue of concepts about teacher education that 
would contribute to the execution of two principal functions. First, 
it should identify appropriate concepts in the literature of teacher 
education, estimating the overlap dr redundancy among such concepts 
and identifying ostensible gaps and the kinds of concepts needed to 
fill those gaps. Secondly, statements about individual concepts 
should give the developer salient and significant infonnation to 
help portray the concept. These statements must define the concepts, 
specify their essential attributes, and give brief examples 'that 
might serve as preliminary sketches for concept portrayal. 

During 1973, a project to develop this catalogue was housed in ■ 
the Graduate Institute of Education, Washington University, under 
the direction of the author. This paper is a preliminary statement of 
the rationale developed to execute the two primary, catalogue functions 

CONCEPTS IN TEACHER EDUCATION 

Concepts; are the building blocks of thought. They outline the 
content of disciplines and undergird their structure. Thus the psy- 
chologist thinks in terms of ^'stimuli" and "responses"; of "rein- 
forcement" and ."extinction"; of "learning," "forgetting," "retention," 
and "interference." For the sociologist , his thinking and work are 
delineated by concepts such as ^^group,'' ^^norm," "interaction," and so 
forth. It is the specialized concepts a group employs to draw atten- 
tion to its issues and phenomena that distinguish its discourse from 
that of anothr group or from laymen. Of course much of the potency of 
a shared and specialized vocabulary lies not only in the individual 



concepts themselves, but in the increased analytical power obtained 
when relationships among sets of those concepts can be established. 
Relationships of this order are to be found in all areas of human 
inquiry that involve logical, prepositional, and theoretical thinking. 
Nor is this generalization restricted to the provinces of the purely 
academic and theoretical. Great strides in the technological base of 
professions such as medicine and engineering have been accompanied by 
vast increments in their level of conceptual and relational sophisti- 
cation. Practitioners are thus able to identify and solve complex 
problems viiich were perhaps incapable even of formulation prior to 
appropriate conceptual and theoretical development. 

Shifting to the field of teaching, and particularly to teacher 
education, we discover that relatively little is available by way of 
a specialized vocabulary or set of concepts about teaching and that 
virtually no prepositional knowledge has been generated. Furthermore, 
individual concepts about teaching are not widely shared among prac- 
titioners. Finally, numerous education scholars have pointed to the 
paucity of technical language available to teachers for discussing 
issues that occur in the classroom (and therefore have some chance 
of defining and solving problems). In short, no language of teaching 
exists that provides a set of intellectual tools by which practitioners 
can isolate areas of difficulty and subsequently comprehend and inter- 
pret them.. Most concepts that teachers possess are not universal and 
may be quite personal. Those teaching concepts that are available are 
often so limited in scope that the individual teacher is left with 
nothing but his own powers of insight and innovation to survive in the 
classroom and to try to improve the instruction he gives his students. 

The author believes this characterization of teacher education as 
a field in conceptual disarray to be substantially accurate. The prin- 
cipal question that we must confront, then, concerns the nature and 
extent of the contribution to be made by a catalogue of concepts in 
teacher education. Clearly, no catalogue of concepts can altogether 
eliminate the conceptual problems of teacher education. It can 
facilitate the process of conceptual clarity, however, if two major 
objectives can be achieved. First, it must specify and delineate the 
definition of the "pedagogical domain of teacher education.*' Secondly, 
it must identify, define, and exemplify appropriate concepts (organized 
according to their functions) within the broader domain of pedagogy. 

PEDAGOGICAL DOMAIN 

An analysis of the phrase '^pedagogical domain** was significant in 
the preliminary effort to organize the catalogue. No clear and widely 
shared referents exist for the term. 

Definition 

We construed it broadly, suggesting that the pedagogical domain 
encompasses those settings and thoce people that bear upon the content 
which is to be taught to pupils, and the methods or procedures employed 
for purposes of conveying content. The domain includes decisions about 
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the choice and communication of content. Thus, not only are concepts 
about the teacher, teaching, or the pupils included in the definition of 
pedagogical domain, but also concepts about relationships between 
teachers and their peers, teachers and administrators, and teachers 
and boards of education and other community groups. Figure 1 portrays 
the major layers of concepts required to exhaust this conceptual 
domain. 

Selection and Delineation 

The pedagogical domain is much more extensive than had been anti- 
cipated prior to our analysis of it, and priorities had to be set to 
guide the selection and delineation of some portion of the domain as 
opposed to its totality. Since teaching practice is more likely to be 
guided by concepts with an ostensible immediate application to the 
daily tasks of teaching than by more remote concepts, the decision was 
made to develop a catalogue of concepts pertaining to interactive 
teaching. This decision was based upon the belief that the preponder- 
ance of concepts most central to the day-to-day activity of teachers 
and pupils are found in that ring. Thus one criterion established for 
the selection of concepts in the catalogue was that they should have 
proximity to the ongoing classroom activities both of teachers and 
pupils. 

Regardless of this decision, concepts that lie within the other 
rings will also, in the future, require conceptual development. For 
excimple, the ring labelled "teacher collegial relationships" might be 
expected to contain relatively few concepts, but they are concepts of 
considerable importance and explanatory power. To our knowledge, 
these concepts have not been systematically collected and analyzed. 
At a guess, however, the concepts in that ring should clarify relation- 
ships among teachers and their peers. Teachers manifestly or subtly 
communicate to one another their ideas on treatment of children, 
maintenance of discipline, amounts and kinds of content, and communi- 
cating content. The intriguing analyses of groups other than teachers 
strongly . imply that the use of sanctions and rewards occurs in all 
sorts of peer working relationships to control both quality and quantity 
of work accomplished and to govern relationships among workers, clien- 
tele, and superiors. Books such as Waller's classic Sociology of 
Teaching^ and the recent monograph Small Town Teacher^ by MacPherson 
suggest that these principles and their component concepts are applicable 
to teaching as well. 

The point of this discussion is to illustrate that concepts about 
interactive teaching, as crucial and as central as they are to the domain 
of pedagogical concepts, do not exhaust it. Nor can any such catalogue 
developed around a fraction of the domain begin to show the integrated 
richness of the domain, as all concept levels influence and are influenced 
by each other. However, with the present catalogue of concepts about 
interactive teaching to serve as a prototype, other areas of the domain 
should be more easily and more quickly developed. 



erJc \ 



55 



Table 1 



A Sample Concept To Appear in the Catalogue: From the Cognitive Section of 
Categories of Content, Reception Learning 

Comparison 



Definition 

A pupil statement indicating the similarities and/or differences 
between two or mrre items. The statement is recalled or reconstructed 
from a prior instructional presentation. 

Indicators 


1. The verbal content of a comparison is similar to that of the 
concept "comparing" (q.v.), which is subclassified under 
productive thinking. The distinction between the two 
concepts is that in reception learning the pupil does, 
not produce the comparison himself. 

2 The term "comparison" is used to represent statements by pupils 
that contain only comparisons, or only contrasts, or which 
may contain both. 

Illustration ' 

As a response to the teacher's question, following a. presentation by 
the teacher on closed plane figures, a pupil says, "A triangle has 
only three sides, but a square has four." Any illustration of the 
concect must include the fact that the original comparison was 
expressed in the teacher's presentation. 

Source 



Smith, B. Othanel, and Milton Meux. A Study of the Logic of Teaching. 
Urbana: University of Illinois Press, 1970. pp. 116-21. 



The decision to use concepts pertaining to interactive teaching 
demanded that we analyze the term "interactive teaching." From that 
analysis ultimately emerged the basic categories of pupil outcomes and 
teacher behaviors that guided the development of the catalogue. Inter- 
active teaching denotes all the events and transactions ttet occur in 
the classroom during the time that the teacher and his pupils are mitally 
engaged in implementing the educational program. Tliese transactions 
may in part be based upon plans and objectives that have been worked 
out in advance, but interactive teaching refers only to the transactions 
themselves. Conceived in this way, interactive teaching occurs within 
a social system composed of four mutually dependent elonents: (a) the 
teacher, (b) the content vhich represents the curriculum or the mate- 
rials to be learned, (c) the pupils as a social group or series of 
social subgroups, and (d) the pupils considered as individuals operating 
within a social contex". Of course, pupils as a group or series of sub- 
groups and pupils as i Uviduals are simply- different perspectives 
adopted for thinking about the same people, but alternative perspectives 
of this sort are quite indispens ible for the teacher. Another qual- 
ification in our delineation of the elements as constituting a social 
system needs to-^be made. When we say -that the elements are mutually 
dependent, we do not meant that they are of equal weight in the system. 
Common observation as well as the weight of research evidence about 
classroom functioning both testify to the major domination of the class- 
room that is built into the teacher's role. The proportion of concepts 
in the catalpgue that reflect what teachers do and how they influence 
pupil outcomes is great. Nonetheless, the teacher' s behavior is dependent 
upon the other elements. If group morale sags, the teacher must restore 
it. If an explanation of a scientific principle is not clear to the 
pupils, the teacher must communicate more meaningfully. If a particular 
body of content bores the pupils, the teacher must omit it, make it more 
stimulating, compel the pupils to endure the dullness, etc. All the 
specific ways in which the teacher's interactive role with pupils is 
modified bv them are virtually countless. 

Although it is appropriate to picture interactive teaching as em- 
bedded in a social system of the kind described here, the interrelation- 
ships and the mutual dependencies among concepts and variables implied 
by such a system far outdistance our current ability to identify and 
define them or to specify their empirical values. The store of pro- 
positional knowledge about interactive teaching is meager. Therefore, 
in developing the catalogue, we disregarded issues of prepositional 
knowledge and the question of vhether the behavior represented by one 
concept is a function of another. Instead we concentrated upon iden- 
tifying and developing concepts about the behavior of the teacher in 
interactive teaching and about the behavior of pupils, either as givens 
or as outcomes, in three major subdivisions: content, groups, and in- 
dividuals in a classroom context. The fourth element, the teacher, 
pervades all three of the major subdivisions. How this scheme generates 
the broad rubrics of the catalogue can be seen in Figure 2. 
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Table 2 



A Sample Concept To Appear in the Catalogue: From the Cognitive Section 
of Categories of Content, Productive Thinking 



Comparing 



Definition 



A cognitive process in which pupils note and indicate similarities 
and/or differences between two or more items on the basis of the 
pupils' original analysis of the i^em. 

Indicators 

1. Comparing often involves a twofold process and results in two 
kinds of products: comparisons and contrasts. 

2. The outcomes of comparing may be very similar to those of 
"comparison" ' (q.v. ).The fundamental differences reside in 
the source of the comparison. In comparing, the product is 
the result of the pupils' productive thinking. Comparison, 
on the other hand, simply involves restatement by the pupil' 
of an intellectual product that he has heard or read. 

3. The character and complexity of the comparison that results 
from comparing may vary in several ways, depending upon 
the structure of the pedagogical control that brought about 
the comparing behavior. For example, such controls, whether 
those of the teacher or of the textbook, may 

a. provide one or more bases of comparison, 

b. provide no basis of comparison, or 

c. provide one item and a basis for comparison and 
request that the pupil provide a comparable or 
contrasting item. 

Illustidtions 



1. Sixth graders have constructed a table to compare two 
explorations. The columns of the table are headed "Columbus" 
and "Apollo 11" respectively. Rows are labeled "size of crew," 
"type of vehicle," "source of energy," "average speed," "distance 
traveled," "cost of voyage," and "significant discoveries." Tie 
pupils are formulating comparisons and contrasts, using the data. 



2. Two of the variations indicated in Indicator #3 above are 
illustrated here. 

a. When asked whether Chicago or Detroit is larger, a pupil 
refers to a population table and responds, "Chicago." 
The control in this case provides a basis of comparison: 
size. Note that the pupil's single word response is the 
end product of the process of comparison. 

b. When asked to compare Hamlet and Macbeth, a student 
responds: "They were both royalty. Macbeth was motivated 
by ambition, Hamlet by revenge. Both were troubled by 

. their consciences. Each was killed, but in different 
ways. . ." Here no basis of comparison is provided^ 

Sources 

Smiths B. Othanel, and Milton Meux. A Study of the Logic of Teaching. 
Urbana: University of Illinois Press, 1970. pp. 116-21. 

Aschner, Mary Jane, and J. J. Gallagher. "A System foi Classifying 
Thought Processes in the Context of Classroom Verbal Interaction." 
Champaign, 111.: Institute for Research on Exceptional Children, 1965. 

Taba, Hilda. "Teaching Strategies and Cognitive Functioning in 
Elementary School Children." Washington, D. C. r U.S. Department 
of Health, Education and Welfare; San Francisco: San Francisco 
State College, 1966. p. 39. 

The major source is Smith and Meux. Aschner and Gallagher classify 
the concept under "Association." Taba discusses "Identifying Common 
Properties" as a process involved in "Grouping." 



The material that appears in Tables 1 and 2 was developed for the catalogue 
by Joseph Gore. 



The bottom branches of Figure 2 indicate the major categorical sub- 
divisions of the catalogue, but in most instances those categories are 
further elaborated. Without suggesting that the catalogue has taxonomic 
properties, we tried to group concepts within the broader categories. 
For example, concepts about pupil groups are organized, with some 
exceptions, according to whether they deal with issues of group morale, 
group productivity, or the control (in a social sease) of groups. 

Concepts pertaining to how teachers control content in interactive 
teaching are grouped depending upon whether they refer to broad approaches 
or styles of controlling content; whether their function is principally 
one of regulating rate, amount, or intellectual level of the content 
flow; or whether the concept involves evaluation of ideas in the con- 
tent, or as expressed by participants. 

COGNITIVE CONTENT CATEGORIES 

A similar principle has been utilized in organizing concepts that 
deal with pupil content outcomes. Following is the rationale developed 
for seeking out and classifying one major dimension of such concepts, 
those having to do with cognitive outcomes that may be achieved by 
pupils. 

Concepts pertaining to cognitive content are construed as indicative 
of objectives about content or as representing pupil content outcomes. 
It is hoped that these concepts will haye two functions: first, to suggest 
to the teacher or prospective teacher arrange of cognitive outcomes 
that appears possible for pupils to achieve and, second, to define and 
illustrate the outcomes carefully in the context of interactive teach- 
ing. In this way, it is hoped, teachers may extend and deepen their 
comprehension of what various cognitive outcomes may look like in pupil 
behavior in the classroom. 

Concepts about content have been subdivided into two principal 
components: reception learning and productive thinking. The reception 
learning component is further subdivided into reproductive learning 
and comprehension. We are indebted to the writings of at least three 
individuals, groups, or schools: Ausubel (1968), 3 Bloom and his col- 
leagues (1956), 4 and Wertheimer (1959), ^ for these labels and the con- 
ceptual structure that guided the development of the component. 

Reception Learning - 

Reception learning outcomes result principally from transactions 
by the learner with instructional materials, including communications 
from the teacher. Ordinarily there is not a great demand that the 
pupil add his original thinking to the ideas or information imparted 
by these materials. On the contrary, the objectives typically require 
him to remember what he has learned. (Ausubel *s conception of reception 
learning and the category identified as "knowledge** in the **Bloom 
taxonom/* are instructive on this score. ) The learner may either para- 
phrase the central elements of what he has been told or he may respond 
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more faithfully to the original information. This type of pupil out- 
comes or responses is called "reproductive" and is included as a first 
classif ici.tion under "reception learning." "Comprehension," on the 
other hand, requires the learner to demonstrate something in addition 
to the simple ability to reproduce a communique previously learned. 

The term "understanding," except for all the ambiguities with which 
it has been charged, begins to approximate our use of comprehension. 
We need to add to it, however, the stipulation by Bloom that the learner 
is acting upon the material. That is, in the language of the Taxonomy 
of Educational Objectives: Cognitive Domain,^ comprehension is reflected 
by the student's ability to translate, interpret, or extrapolate from a 
given body of material. We have chosen to classify comprehension under 
the broader rubric of reception learning because of the dependence of 
such outcomes upon a given message, passage, or coiranunicat ion , The 
learner is not asked to create what is essentially a new product. In- 
stead his actions reflect his ability to develop in a fuller sense 
what the meaning and implications of the message may be. 

The importance of concepts about reception learning class if ed under 
interactive teaching is obvious. Without wishing to guess at percentages 
it is apparent that a very large proportion of interactions in the class- 
room that deal with content do so at the level of reception learning as 
we have defined it. Thus, the frequency with which teachers will gen- 
erate and encounter behaviors of these types and tlieir pervasiveness 
in classroom interaction constitute important criteria for their in- 
clusion. 

Prod?:ctive 7'lunking 

Productive thinking occurs, no doubt, with significantly less fre- 
(|uency than does reception learning. The principal criterion used to 
include concepts about this area in the catalogue is that pupil out- 
comes of productive thinking are vital in the achievement of most 
educational objectives pertaining to the improvement of students' 
intellectual skills and abilities, and the more teachers learn about 
such outcomes as they are manifested through interactive teaching, the 
more likely they are to iJentif}' appropriate pupil behaviors when they 
occur, vnd tc arrange a classroom that is conducive tx) increi«.sed 
productive chinking by their students. Although we do not wish to 
stretch the similarity too far, there are some parallels between 
reception learning and productive thinking. For example, reception 
learning includes the concept of ''explanation.'' According to this 
meaning, the learner would be able to explain an event, process, or 
phenomenon when the explanation had been given to him. Although not 
original in any fundamental sense, the explanation would f.resumably 
be complete and accurate. One kind of behavior involved in produc- 
tive thinking is ''explaining." In this case the student is confronted 
^with an event, and although detaij^^^bout how much direction he is 
given may vary, he is given something less than a full explanation. 
The construction and communication of an explanation become his 
principal intellectual tasks. ouchmann*^ provides excellent illustrations 
of explaining. Pupils, in this case in intermediate grades, are shov/n 
simple physics demonstrations. These brief films, end with the question. 



"IVhy [did such an event occur]?" The process through which children 
learn to provide explanations is a lengthy and complex one, but the point 
is that in the. end, the learner must construct the explanation. One 
is not given to hijn. This is exactly the case for what we call pro- 
ductive thinking. 
•* 

PrQdticifig the Concept s 

We mentioned earlier that much of the conceptual argument for the 
oj-ganization of this section of the catalogue came from the writings of 
'Ausubel, Bloom, and perhaps others. Similarly, for the concepts that we 
identifed and defined in the body of the section, we are indebted to a 
group of investigators who have studied interactive teaching in ongoing 
classrooms and whose fundamental attention has been to what we call the 
cognitive components of such interaction. The scholars to whose work 
we refer include, but are not restricted to. Smith (1962, 1967), ^ 
Bellack and others (1966), 9 Taba (1964, 1966), 10 and Aschner and 
Gallagher C1963, 1965). ^ These works have provided us with valid 
models of cognitive teacher-pupil interaction. 

Our task in producing concepts for this section consisted of 
several elements: First was the need to construct a conceptual frame- 
work to help identify concepts to be included in the category and 
organize the presentation of those concepts. Secondly, we had to com- 
pare concepts across systems (e.g., between Smith and Bellack) and make 
an initial effort at integrating these concepts rather than simply col- 
lecting them. The latter tactic would have contributed little to an 
integrated view of concepts about teaching. The tactic we chose, of 
attempting a preliminary integration of concepts with an eye toward 
reducing the number to be developed, has been successful only to a 
limited degree. Such efforts, however, seem to be desperately needed 
in discussions of education, and we would hope that subsequent workers 
would follow the spirit of our intention and surpass the failings of 
our execution. Finally, our effort included attempts to define, de- 
lineate, and illustrate the various concepts settled upon. We intended 
to provide a maximum amount of be^^innine assistance to the materials 
developer or the general teacher educator who wishes to expand upon 
these concepts and incorporate them into the curriculum of teacher 
education. This latter effort has been materially aided by the exis- 
tence of countless examples and illustrations included in the mono- 
graphs about interactive teaching to which we earlier alluded. We 
have drawn freely from these materials, and in each case we have cited 
the sources, partly as an acknowledgement, but also as a reference for 
the user to consult the original material for additional examples or 
meanings beyond those used in the limited confines of the catalogue. 

FORMAT OF CONCEPTS WITHIN THE CATALOGUE 

The preceding discussion has focused on the chain of reasoning 
used to limit and define the conceptual domain represented in the 
catalogue. By defining the domain, we included the specification of 
the concepts wivich it contains. We now turn to the descriptions of 



concept in the catalogue. Once the developer of protocol materials 
has located the concept (s) he wishes^^to develop, the dimensions and 
magnitude of the problems confronting h.^n undergo a shift. He must 
be concerned with the production of protocol materials or what has 
earlier been referred to as **the portrayal of concepts** C^iudgins, 
1972). 12 In that monograph, three basic issues that the developer 
must contend with as he translates abstract labels into ^vital proto- 
col materials for teacher education were established. The first two 
were salient to the format adopted for descriptions of concepts in the 
catalogue; the third remains essentially a matter to be determined by 
individual developers. The issues referred to are (a) analytic issues, 
(b) didactic issues, and (c) issues of outcomes. 

.A nalytic Issues - ' 

Analytic issues pertain principally to the clarity and comprehen- 
siveness of statements of the concept. Attention to concept analysis is 
a first and critical stage in the production process. Little or no 
attention is paid to arranging conceptual displays for purposes of 
teaching; rather, emphasis is on explaining as fully and as clearly 
as possible the meaning of the concept and including the elements or 
attributes necessary to ensure its comprehensiveness. The definition 
and the presentation of indicators contained in the descriptions of 
each concept included in the catalogue attend to the analytic issues. 
Tables 1 and 2 illustrate two concepts in the Content dimension of 
the catalogue.' The first concept, ^'comparison,** assumes only recall 
by the pupil of what he has previously heard or read. In '^comparing,** 
pupils must act upon a given body or item of consent in order to draw 
from it explicit statements about similarities and differences between 
it and some other event or phenomenon. 

Didactic Issues 

Didactic issues involve the developer in considerations of por- 
traying a concept in order to maximize its ef fective/^ess as a teaching 
material. At a minimum,, didactic issues demand that che developer 
present scenes or episodes that are cieai and relatively siniple examples 
of the concept, S^nce the concepts with Which teachers have to deal 
in class are seldom so simple, the developer ^:]^dinarily needs to work 
out strategies for producing lifelike protocol materials. A rationale 
Tor this procedure and suggestions for achieving didactic success are 
treated extensively in Hudgins (1972).^^ In the cataglogue, didactic 
issues are treated by one or more brief illustrations of the concept 
in a classroom setting. When cohfusion with closely related concepts 
appei^rs imminent, examples have been added to aid discrimination 
between the two. These examples are not intended to be prescriptive 
or in any way limit the conceptual or artistic freedom of the devel- 
oper. Presumably most of these concepts could he illustrated in almost 
limitless ways. The intention is simply to portray faithfully the 
concept in one or two ways. 

Analytic issues and didactic issues go far beyond tlio introduction 
to them included in our descriptions within the catalogue. They play 
important and continuing roles throughout the developmental process. 
On the other hand, the descriptions that we have provided ought to 
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assist the developer in his preliminary consideration of these issues 
and in his firzt attempts at portraying the concept. 



Finally, in most instances, we have cited the major references or 
sources consulted in the development of illustrations. The developer 
should be materially assisted by having a brief but highly pertinent 
set of references for more ,extensive analytic and didactic guides. 



ASPIRATIONS FOR THE CATALOGUE: ELABORATING THE CONCEPTUAL WPS OF 
TEACHERS 

We hold two major aspirations for the utilization of this catalogue 
of concepts about interactive teaching. First, if protocols of the 
approximately one hundred concepts in the catalogue are eventually produced, 
a rew, systematic, and potentially powerful set of instructional mate- 
rials will be available to teacher educators on a wide scale. J.f the 
concepts that our analysis of the domain of interactive teaching has 
led to are productive in that they help teachers in training or in- 
servicc to more fully understand and interpret significant events as 
they occur in the classroom, then the basic purpose for which the 
catalogue pro j ect-- Wds originated v;ill have been adequately achieved. 
The cataloguev/itself is an inert object, and the viability of that 
aspiration can be tested only when a sufficiently large proportion of 
the concepts identified therein have been developed and utilized in 
-ongoing programs of teacher education. Whether such development is 
to occur must, of coui:se,vbe decided by others. We would hope that 
the catalogue will Stimulate rather than inhibit such development. If 
it does, the catalogi*e may make some contribution toward helping to 
extend and deepen the conceptual base that teachers have for the ex- 
planation and interpretation of classroom events. 

The second aspiration hais a broader scope and involves two ele- 
ments: on e^ pertaining to the utilization of the catalogue and the 
other to the interpretation of the. catalogue as a stimulus to the gen- 
eration of concepts. Throughout this discussion we have attempted to 
comment upon the applicability of the .catalogue to the work cf protocoi 
developers. However, the conceptual framework of the catalogue as well 
as the content: of individual concepts^may also have applicability for 
many teacher educators who are not formal materials developers, but who 
are responsible for the construction of teacher education programs and 
courses. Finally, the catalogue is not intended to be used in a purely 
prescriptive fashion, despite the fact that great thought and effort 
have been poured into both the classificatory system and the development 
of individual concepts. The ooncepts identified are important .and de- 
fensible ones for delineating the interactive teaching subdomain; how- 
ever, we have no basis to certify that the concepts so identified 
exhaust the subdomain or the categories which we have Established within 
it. Were our work to fepur others on to extend what we have begun, or 
even to supplant it with better reasoned arid more productive formu- 
lations of the domain^ we would be happy to conclude that our efforts 
had contributed toward the more intellectually rigorous education of 
teachers which is, after all, the fundamental objective which all of 
us share . ' - • 
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HISTORY 

\personnel training for the production o£ protocol materials in 
teacher education began in 1970 with support from the Bureau of Educa- 
tional Personnel Development (BEPD) of the U.S. Office of Education 
(USOE). The first meeting with nrotocol project directors was held 
in Chicago during the summer of that year and chaired by B. Othanel 
Smith whose description of protocol materials in Teachers for the Real 
World^ stimulated interest and support that enabled the program to 
begin. Don Dav^es and Don Bigelow from USOE made presentations at 
this meeting and added the weight of^;^heir convictions favoring the 
development of protocol materials to the initial plans in the program. 
Doris Gunderson, BEPD program director, assisted in establishing 
procedures to be followed by project directors. 

Three groups were combined to provide the support necessary to 
develop protocol materials. These three groups were BEPD, the Leader- 
ship Training Institute on Protocol Materials (LTI] , and the project 
directors. The interdependence of these three groups proved to be 
essential to the success of the program. A brief description of each 
supplies the foundation and context within which the program operates. 

BEPD supported the training of personnel in the production df 
protocol materials as part of a comprehensive design to improve the 
performance of educational personnel. Teacher performance is dependent 
on. a teacher's knowledge and his ability to employ his knowledge. Pro- 
viding materials that decrease the dissonance between vicarious learning 
and contact with pupils aids in the acquisition of knowledge. Textbooks 
provide instruction about teaching, but they cannot actualize behavior 
for study. Classroom behavior provides actual behavior for study, but 
it is unpredictable, vanishes after it occurs, and is impossible to 
study completely from memory. A compromise between these two appi-^oaches 
is offered by protocol materials. 

During the spring of 1970, BEPD screened responses to requests for 
proposals. The 11 sites designated to train personnel and develop pro- 
tocol materials began their work that fall. Since the first year of the 
program, support from BEPD and, more recently, the National Center for 
the Improvement of Educational Systems (NCIES) has continued; briefings 
on protocol materials have been given to personnel at USOE. LTI has 
received continued support to service the program, promote the training 
of personnel, and disseminate and publicize prcftocol materials to advance 
personnel training throughout the country. As LTI expanded its respon- 
sibilities, significant support for the program continued from the 
program director, Doris Gunderson, and from other USOE officials, par- 
ticularly William Smith and Allen Schmieder. 

LTI wan formulated to give technical assistance to the protocol 
projects, provide liaison between projects and BEPD, coordinate the 
protocol projects, and relate protocol materials to other federally 
funded training programs. It was created when the projects were funded 
and is directed by B. Othanel Smith. Based at the University of South 
Florida in Tampa, it utilizes personnel from throughout the nation whose 
training and experience qualify them to give technical assistance. 2 
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The major activities of LTI have been to plan the overall development 
of the program; meet with project directors to discuss and solve pro- 
duction problems; arrange consultant services for project directors; 
and assume, responsibility for standards of production, publicity, 
training of additional personnel, field testing, and development of 
dissemination procedures. 

One of the original 11 sites designed to train personnel was the 
Florida State Department of Education. Its responsibility was different 
from that of the other 10 locations. ^ During the first year, the Florid 
project was responsible for producing protocol materials and forming a 
consortium of institutions of higher education whose combined efforts 
would produce and field test the materials. The production of protocol 
materials through a consortium proved to be unrealistic, but the Florida 
project continued to serve as a field test site for the other projects. 
The project also trained personnel in the public schools and the insti- 
tutions of higher education in field test procedures and the use of pro- 
tocol materials. 



MAJOR DEVELOI^IENTS 

Considerable progress has been made since -the beginning of the 
program. The issues and answers that arose can be understood best by 
those who were engaged in the program. However, a description of the 
major concerns and their solutions can give some idea of the progress 
in development of techniques to resolve production and traininj problems 
and the procedures that grew out of the experience of those involved in 
the program. Following is a discussion of the major developments in the 
program including (a) the definition of protocol materials and the 
selection of conce'pts, (b) technical requirements and criteria, (c) 
supplementary materials, and (d) field testing and dissemination. 

Definition 

A general definition of protocol materials served to guide those 
who submitted proposals to BEPD. The basis on which the program began 
can be found in the following statement: 

Until the development of educational technology, it was dif- 
ficult to reproduce teaching behavior. But today audio and video 
recordings of behavior can be made and studied in detail. They 
make it possible to teach theoretical knowledge of pedagogy in 
the context of its use as well as in formal courses. 

To folJow this mode of instruction it is necessary to have 
available an extensive supply of audio and video recordings of 
home, street, playground and classroom situations, of committee 
meetings, and interviews. These recordings will be referred to 
here as protocol materials, behavioral situations, or sinply 
situations. Protocol materials should represent the most poverty- 
stricken and most affluent rural and urban communities, as well 
as all minority groups. They should also represent all grade 
levels and teaching procedures such as problem- so Ivin g, question- 
and-answer, and group discussion.^ 



Refinements in this description were needed before production could 
actually begin. Based on the definition that a protocol is. an original 
draft, minute, or record of a document or transaction,^ protocol mate- 
rials are records of behavior portraying school events. The developers 
had to obtain a record of behavior on film or some other appropriate 
medium that would give the viewer an episode to further his conceptual 
knowledge. This task introduced numerous questions requiring precise 
answers. 

Development Procedures 

The developers and LTI followed certain procedures. The behav- 
ior illustrating the concept in each protocol occurred frequently 
enough to enable the observer to see numerous behavioral indicators of 
the concept. The number of unrelated behaviors that might distract the 
viewer from seeing the intended behavior was held to a minimum. Each 
episode lasted 5 to 10 minutes to economize the production and instruc- 
tional and viewing time. Although the episodes were to be a **slice of 
reality,*' some staging and scripting was allowed to guarantee that the 
behavior illustrating the concept would occur. It was initially regarded 
as inappropriate for protocol films tb> include any pointers or cues to 
help the viewer with his observation. Tliis requirement was eventually 
relaxed to allow experimentation in determining how much help should be 
given to a viewer. In some cases, numbered frames were allowed as 
references to parts of a film. Under no circumstances was it acceptable 
to give the viewer a descriptive narrative on the behavior depicted. A 
narrative would remove the burden of interpretation from the viewer and 
violate this fundamental principle of protocol materials. Within these 
general operating procedures, the protocol developers were able to 
proceed with planning and producing protocol filins. 

Interpretive Role of Teacher 

A problem arose in respect to the concepts to be exemplified in 
protocol films. A teacher should be able to make accurate diagnoses 
if he is to cope effectively with the events he faces. Concepts are 
the organizing tools that enable the teacher to classify behavior or 
events according to a given type. 

Suppose that a student throws an object and subsequently wanders 
around the room. How are these actions to be understood? They 
could be indications of either attention seeking or **critical dis- 
sension.** The teacher will not understand these acts correctly 
unless he is able to classify them correctly. If these actions 
are associated with others that are clearly in the category of 
attention seeking, the teacher is apt to be correct if he clas- 
sifies these actions in that category also. On the other hand, 
they are apt to be signs of critical dissension if they occur 
in a context of other acts that clearly belong to this particular 
class. In any event, the teacher understands behavior by 
classifying it, and the accuracy of his categorizing will deter- 
mine the correctness of his understanding and partly the adequacy 
of the subsequent treatment.^ 
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This emphasizes the interpretive role of the teacher and points out 
the advaiitage the teacher has if he can diagnose and classify behavior. 
Protocol materials should provide the observer with examples of behav- 
ior that will help him acquire the conceptual knowledge necessary to 
interpret pedagogical events. 

Most preservice instruction on concepts has been conducted in the 
lecture hall or through discussions describing instances to illustrate 
a concept. Students who learn about concepts this way may do very well 
on paper-pencil tests; however, when confronted with the behavior in the 
classroom, they are often unable to classify and accurately interpret 
it. This inability of the beginning or student teacher to interpret 
accurately results in the complaint that there is little relationship 
between theory and practice. Actually, there is a close relationship 
between the two; the usual methods of relating them have simply been 
ineffective. Words alone are inadequate to describe behavior. 

Protocols vs. Observation 

The protocol materials program supports the premise that if behav- 
ior could be captured on film, students in training could see the 
behavior exemplifying a given concept and be better prepared to rec- 
ognize and classify behavior. Why .bother filming behavior when the 
same behavior can be observed first hand in the classroom? Protocol 
materials offer some advantages not found in^direct classroom obser- 
vation. Anyone who has faced the logistical problems in arranging and 
supervising classroom observation experiences must ask if it is a worth- 
while use of the time and efforts of the university and public school 
personnel. 

Apart from the administrative details that accompany such arrange- 
ments, there are other even more important factors. If an observer 
visits a classroom to observe specific activities, there is little 
assurance that the given activities will take place. It is not uncom- 
mon for students who visit a school not to see a typical class because 
of scheduled standardized testing, a school assembly, or a disruption 
in the expected lesson of the day. The time spent in travel, sitting 
through a school assembly, or watching students complete an examination 
is all too frequent to rely on observation as a predictable part of the 
teacher training program. Additionally, even though the school may be 
operating normally, events that are productive for the observer may not 
actually occur. The phenomena an observer wishes to see in the behav- 
ior of the teacher or pupils may not be portrayed during the time of 
the visit. And even if the observer is in the classroom when certain 
behavior occurs, being untrained, he may miss it. If he does see it, 
the behavior vanishes as soon as It happens; it cannot be restaged for 
further study and analysis. 

It is apparent that theory taught in the college or university is 
not very useful" to most students who attempt to use their knowledge in 
the classroom. Student placement as observers or student teachers 
offers no assurance that the events they encounter will serve any pur- 
pose for their training and eventual teaching responsibilities. Today's 
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teachers capable of making proper diagnoses have acquired their ability 
through time, exposure to the events in the classroom, aml^ interpretation 
of behavior by trial and error. Protpcol materials are designed to 
enable the beginning teacher to be better prepared to interpret and 
diagnose behavior. In addition, protocol m.aterials are beneficial to 
the experienced teacher who has not been sufficiently analytical or 
sufficiently schooled in interpreting behavior. Protocol materials 
should enable preservice and in-service teachers to acquire a better 
organizing network of concepts, the vocabulary necessary to describe 
behavior, ar.d the observational and diagnostic ability to make accurate 
interpretations of behavior. 

Master Plan 

A master plan was d3veloped to guide the protocol program (see 
Appendix in Cruickshank' s paper). The plan provided for concepts to 
be selected in the pedagogical domain and in the basic fields of study. 
About one- third of the first projects were in the basic fields, the 
remainder in the pedagogical domain. The master plan provided for con- 
cepts to be selected, in different settings such as the classroom, school, 
or community; different age levels such as early childhood or adolescence; 
and different sources^ of behavior such as the teacher, pupil, or teacher- 
pupil interaction. JJt was recommended that the classroom setting 
receive highest priority and that the developer be free to select the 
source of behavior and the level he preferred. 

The master plan gave broad direction to the program; the specific 
concepts to be portrayed remained to be chosen. Shortly after the 
program began, it became evident that the literature on concepts is 
vague; more precision and rigor were needed in naming and defining con- 
cepts. The definition of concepts being prerequisite to the production 
of materials, LTI concentrated on recommendations to help in the devel- 
opment of workable definitions. 

Concepts are not all of the same order. Different definitional 
forms are necessary to account for the variety of concepts that can be 
portrayed in protocol materials. In developing protocol materials, it 
was suggested that concepts be expressed in one of four possible forms: 
classification, equivalent-expression, open-context, and conditional. 
The classification form of a definition provides for the association 
of a concept with a category and discriminating criteria to distinguish 
one concept from others^ In the equivalent-expression form, a concept 
is set forth by an expression that is equivalent to the word or words 
used to name that concept. liic open-context form expresses concepts 
whose definitions are imprecise because the boundaries of the term are 
not rigidly limited. The interpretation of behavior may vary according 
to the conditions that precede the behavior; the conditions are a part 
of the definition of a term, and the conditional form then becomes 
appropriate . 

Sel ection of Concepts 

The developer was confronted with the question of whether he should 
first film behavior and then determine the concepts that would provide 
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for the interpretation of the recording or if he should choose the con- 
cepts first and then control situations to elicit the desired behavior. 
The developer chose to select the concepts first to assure success in 
their recordings. (See ^'The Protocol Materials Movement: An Exemplar 
of Efforts To Wed Theory and Practice in Teacher Education" by Cruick- 
shank, 9, for another point of view.) 

In selecting concepts, the developer first entered the master plan 
and selected a level, setting, and source of behavior in which he wished 
to portray behavior. His second step was to name the concept he planned 
to exemplify. Finally, he defined the concept according to one. of the 
linguistic forms provided. Subsequent steps remained for the developer 
to specify the behavior required for the protocol, communicate his 
decisions to the technical staff, and arrange all the logistics and 
editing that remained. All of his later decisions were related to the 
work done in selecting and defining concepts. 

The project directors were given considerable latitude in selecting 
the concepts which they wished to portray. Each project had listed the 
concepts it intended to illustrate, but the' level of understanding at 
the time the proposals were prepared and the improvement in that under- 
standing after the projects were underway necessitated revision of the 
original proposals in many instances. In some cases different concepts 
were chosen; in other cases the selected concepts were modified either 
in respect to their definition or in the number of concepts to be in- 
cluded in the final products. Project directors were understandably 
ambitious in estimating the amount of work they could complet?i; more 
realistic plans emerged as they gained more experience in the production 
of materials. The concepts selected were originally those concepts that 
most nearly coincided with the developers' own background and knowledge. 
Eventually the project directors related the concepts they chose to a 
taxonomic or nomothetic system and sharpened their definitions to attain 
a clearer understanding and distinction between the concepts they were 
illustrating and other concepts that were related. As work p;rogressed, 
the initial choices held up well, but the amount of clarifyilf| and 
support information required increased considerably. After several 
years in the project, developers have learned to identify concepts 
that are part of a system rather than isolated ones. 

Standards 

Tne LTI committee on field testing, utilization, and dissemination 
met with four project directors selected by their peers to determine 
the standards for protocol materials. The committee was to supply stan- 
dards of acceptability to guide directors and to protect the users of 
protocol materials from inferior products. Requirements developed by 
the committee included technical and educational standards. The tech- 
nical criteria selected were related to audio quality, visual quality, 
instruction, format, and copyright procedures. The educational criteria 
applied to standards of concept attainment, learner reactions, and user 
satisfaction. 

As far as technical quality was concerned, the developer was 
advised to use a double-sound system and to produce audio quality at 



such a standard that fewer than 10 percent of the viewers would report 
unfavorable perceptions. The visual quality was required to have such 
sharpness of detail that virtually any viewer under normal classroom 
projection conditions vould be able to distinguish the critical features 
of the images presented. The producer was advised to protect his first 
answer print by preparing a master from the conformed original for 
duplication of subsequent prints. The format recommended depended go 
the nature of the concept, but the acceptable media listed were 16mm 
film or EBR transfer film, slides in 3Smm or filmstrip, and cassettes 
rather than reel tapes for audio protocols. Each package was to in- 
clude a rationale for the use of the media and guides for instructors 
and students. The guides were to contain suggestions' for using pro- 
tocols, ways for viewers to record observations , means to measure 
acquisition of concepts, and recommendations for relating material to 
prior learning and practical situations. The instructional package was 
to be organized and packed for easy storage and shipping. 

The recommendations about copyright provisions were that written 
releases should be obtained from all persons and for all materials used 
in the protocols in addition to written releases for background i^^usic, 
sound effects, lyrics, and other audio materials. The usual copyright 
application and notice was listed as a requirement for protection of 
the overall design and treatment of the material and its copyrightable 
components. 

The educational criteria specified that 80 percent of the learners 
completing the unit be able to identify 80 percent or more of the 
examples of the concept. Since it is difficult to establish criterion- 
referenced standards with new materials, an alternate standard was given. 
A statistically significant difference in favor of the performance of 
groups who studied concepts with protocols was required in comparison 
with groups who studied the concept under conventioi""al methods. Learner 
reactions should be shown through field testing to be positive to the 
extent that fewer than 10 percent of the learners would report unfavor- 
able reaction to the protocol unit. Learner reactions were to be 
obtained on the interest level of the materials, perceived relevance 
to teaching, desire for additional protocol materials, and willingness 
to recommend protocol materials to other students. The standard of user 
satisfaction was aimed at getting information from the instructor using 
protocol materials in his work with pupils. Field test data were re- 
quested particularly the users' reaction to the importance of the concept 
the ease of using protocols in the instructional program, and the appro- 
priateness of the concept to be taught- 

As a consequence of the work of the committee on protocol standards, 
the developers had guidelines at their disposal that would enable them to 
tailor their production to criteria for acceptability. The users were 
assured field test information about the protocol materials that would 
help them make a judgment about selection and utilization of protocol 
materials. 



Supplementary Materials 

In addition to the record of behavior, the protocol package contains 
written material: teacher guides, pupil guides, and general information. 
One issue at stake in the preparation of supplementary materials was 
whether the producer could impose on the eventual user the instructional 
method to use with the protocol. The user can ignore the written mate- 
rials and use a film or audio tape according to his own preference. In 
the case of protocol materials, the user may regard behavior as exemplary 
and use a film to instruct in teaching skills. Or, since the behavior 
. recorded may be considered good teaching technique, the user may employ 
the protocol materials as training materials. There is no guarantee 
that the user will not use the materials to improve behavioral abilities 
rather than to improve interpretive abilities. Another consideration 
in the use of supplementary materials was whether a given instructional 
method should be reconm ended. Some instructors may prefer a didactic 
approach; others may favor an inductive or inquiry approach; and "^till 
others may f^el that a heuristic approach should be utilized. The ,de 
veloper may provide materials according to one bias, and the user may 
use the materials according to another. A third issue was how to assure 
that the instructor would read the printed materials, benefit from the 
experience of the developer, and use the approaches recommended for in- 
structing with protocol materials. None of these issues were adequately 
resolved. 

It was recommended that the developer supply written materials 
giving the evidence from field testing and present the procedures used 
aiid recommended by the developer. In some cases, the developer might 
suggest several approaches for the user to employ. For example, he 
might recommend that the" protocol materials could be used in large- 
group, small-group, or individual instruction according to the dis- 
cretion of the instructor. He may have field test data to show that 
any organization of the students will yield about the same results; 
the user should not feel bound to a given approach. In other cases, 
a varied sequence might be appropriate. However, the only obligation 
the developer can have for the eventual use of his materials is to 
present the information- about procedures he used, present the field 
test data from that procedure, and suggest to the user that any 
deviation from the recommended procedure might yield different results. 
The developer must make supplementary materials readable, brief, and 
comprehensive. Brevity should encourage the user to read them and 
comprehensive coverage should assist in answering questions about every 
aspect of the use of the materials. 

Field Testing 

A requirement imposed on project directors was that all materials 
should be field tested at two levels before they were made available 
for general consumption. One level of field testing, completed under 
the direction of the producer, was primarily to assist in the production 
of the materials. If local field testing revealed that the criteria 
established for acceptable materials were not being met, then the devel- 
oper could revise his product until it met the standards. The second 
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level of field testing, under the direction of the project in the 
Florida State Department of Education, provided trials cu neutral sites 
and in a variety of settings giving an objective report on the standards 
in universities or public schools at the preservice and in-service level. 
After completion of tield testing^ in Florida, changes in the product 
were completed prior to release. After all standards are met, the pro- 
ducts are ready for dissemination. 

Dissemination 

The dissemination of products is an unfinished task ?t the moment. 
Part of this effort is to inform the market of the existence of protocol 
materials. LTI and the developers have shared responsibility for pub- 
licizing their work. Presentations about protocol materials have been 
made at the annual meeting of the American Association of Colleges for 
Teacher Education and the American Educational Research Association. 
Several workshops have been conducted by LTI or project directors at the 
state, university, and local level. Until projeicts were ready for use, 
publicity abo.ut their production was premature- Thus, the first two 
years of production were completed without much publicity. Now that 
the program has been underway long enough for some products to be com- 
pleted, the mechanics of dissemination are being dev-e-loped. 

CURRENT STATUS 

During the 1973-74 academic year, the project directors have three 
major tasks. First is to complete the remaining details so that all 
products underway are finished. Some products are in final form and 
are being distributed; others are too inadequate to salvage; and others 
are nearly completed but require some additional work to meet standards. 
The second task is the establishment and implementation jf procedures 
to disseminate products. The dissemination process is just beginning. 
It may be necessary to publicize the work further in order to instruct 
users in the employment of protocol materials and to contact profes- 
sional organizations, state departments, universities, local schools, 
and other federally funded projects. Finally, developers rust continue 
with the production of the additional materials they have contracted to 
produce. 

Most of the protocol materials have been developed in the peda- 
gogical field, and the concepts chosen have been selected by the indi- 
vidual developer. An expansion of both of these efforts is taking 
place. Nine additional trainees have been chosen from the fields of 
mathematics, social sciences, and English to learn how to produce protocol 
materials and to compile a list of appropriate concepts for protocol mate- 
rials production. At the end of the current year, tnese trainees should 
be able to carry on the development of protocol materials in their basic 
fields of study. They should also produce a catalog of appropriate 
concepts for the preparation of teachers in their fields. A subcontract 
has been used by LTI to organize the concepts in the pedagogical field. 
Consequently, the selection of concepts for protocol materials production 
should become more systematic and be based on concepts of greatest need 
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and importance to the teacher. This is not to suggest that concepts 
that have been developed are not especially useful; it only rsans that 
a systematic study of the field was lacking at the inception of the 
program. 

Consderable improvement in production techniques has occured. 
Puture developers will benefit from these early experiences. A list 
of concepts in the pedagogical domain will increase the usefulness of 
the concepts selected for protocol development. The training of 
additional personnel and the organization of the concepts in the basic 
fields of study will provide the foundation for expansion of pro- 
tocol development across a wider spectrum of the total field of teacher 
preparation. The efforts at dissemination that will be carried forth 
should yield infonnation that will open effective channels for distri- 
bution and use of protocol materials. 



SUNMARY 

The protocol materials program has come on the scene at a fortu- 
itous time. The schools have failed to capitalize on the technology 
and research potential that would improve them, but there appears to be 
a receptive climate for remedying this lack. The emergence of the 
Nat ional . Institute of Education (NIE) is one concrete illustration of 
recent change in promotion of research in teacher education. The ade- 
quacy of NIE remains to be seen, but clearly the potential to start a 
comprehensive national effort in educational research is at hand. The 
miniaturization and the refinements in recording behavior make the 
potential of technology in the schools plausible and effective. Pro- 
tocol materials provide the means to capitalize on these research and 
technical developments and employ them in the development of new mate- 
rials for the instruction of preservice and in-service teachers. 

\ Another illustration of the timeliness of the protocol program is 
that the schools are faced with demands for iccountability such as 
the movement towards competency-based certification of classrom teachers. 
One of the critical areas of competence for the teacher is his inter- 

pi\etive ability to diagnose situations. Protocol materials provide 
the means to assess teacher competence in the conceptual knowledge he 
possesses. 

When the protocol materials program began, it .was impossible to 
anticipate the numerous problems that would arise. The project direc- 
tors and LTI have combined forces with federal support to solve most of 
the major ones. Protocol materials meet standards that protect the 
user ;and guide the developer. The analysis of concepts and the rigorous 
demands placed on the developer to clarify his work is bringing better 
order to the pedagogical and basic fields of study. Materials that pro- 
vide research opportunities and also respond to the demands for teacher 
competence are now being developed in the program. The training of 
teachers can be carried out with protocol materials as a bridge between 
theory and practice heretofore not possible. 
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The future of protocol materials development is dependent on ade- ^ 
quate financing, successful use of materials, and a cadre of people who V 
are willing to devote their time and energies to this difficult assign- ^ 
ment of recording behavior under rigorous standards. The program has 
proven that it is possible to produce protocol materials. Field test 
data reveal that the materials are effective. The additional knowledge 
about the preparation of teachers is a substantial contribution. The ^^"^n^^XA 
time is ripe for building on the foundation that has been laid and de- 
veloping a collection of protocol materials that will elevate the 
training of teachers to the higher level that the current- state of ' 
knowledge makes possible. 
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NOTES 

1. B. Othanel Smith and others. Teachers for the Real World (Wash- 
ington, D.C. : American Association of Colleges for Teacher Education, 
1969). 

2. LTI Membership: Othanel Smith, director; Don?ld E. Orlosky, 
associate director. Panel Members: James A. Banks, Jam^s M. Cooper,. 
William Druinmond, Ned Flanders, Vem Haubrich, Charles J, Modiste, 
Irving Morrissett, Allen C. Purves, Frederick A, Rodgers, H. Del 
Schalock, Charles Schuller, Charles Scruggs, Roger Shuy, and Richard 
L. Turner. 

3. The locations and directors who a ssumed responsib^ lity during the 
first year of production were David Berliner, Far West Laboratory; 

Paul Twelker, Teaching Research, Oregon State System of Higher Education; 
Celeste Woodley, University of Colorado; Bryce Hudgins, Washington 
University; David Gliessman, Indiana University; Judith Henderson and J. 
Bruce Burke, Michigan State University; Donald Cruickshank, Ohio State 
University; William Heiner, Bucknell University; Peter Dow and Anita 
Mishler, Education Development Center; Richard W. Lid, San Fernando 
Valley State College (now California State University at Northridge); 
and Pauline Mastersbn, Florida State Department of Education. 
During the second^ year, Walter Borg at Utah State University, Theresa 
Love at Southern Illinois University, and Patricia Hef feman-Cabrera 
and William Tikunoff at the University of Southern California were 
added. Washington University of St. Louis discontinued after one 
year due to the difficulty in acquiring adequate media production 
personnel. Throughout the program some of the original personnel have 
been replaced; currently, the directors listed above have continued 
with the following exceptions: R. E. Myers at Teaching Research at 
Oregon, Frank Zidonis at Ohio State University, Edward Martin and 
Susan Bernstein at Educational Development Center, Greta Morine and 
Gloria Golden at the Fc\r West Laboratory, and G, Micbafel Kuhn at the 
Florida State Department of Education Project. 

4. Smith, op. cit. , pp. 52-53. 

5. Webster's New Collegiate Dictionary (Springfield., Mass. : G. 5 C. 
Merriam Co., 1973), p. 927. 

6. B. Othanel Smith and Donald E. Orlosky, rhe Development of Pro- 
tocol Materials , Acquiring Teaching Competencies Report No. 3 (Bloom- 
ington: Indiana University, School of Education, January 19 73), p. 12. 
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A Protocol Materials Evaluation: The 
^ Language of Children 

by Victor M. Rental 
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INTRODUCTION \ 

. in April 1970, several faculty members from the Ohio State Mniver- 
sity College of Education discussed the possibility of responding to a 
U.S. Office of Education (USOE) Request for Proposal for aT^series of 
protocol materials. 1 Few of us had more than a slira notion of what 
protocol materials were, our only knowledge coming iiiainly from Teachers 
for the Real World. According to this, protocol materials are visual, 
auditory, or printed reproductions of relevant situations occurring in 
an educational context. 3 These episodic records provide teachers in 
training with the opportunity to observe and. study classroom situations 
against a backdrop of educational theoryl Repeated observation and 
interpretation of these slices of classroom life, according to Smith, 
should lead to the acquisition of concepts which serve a basis for 
interpreting new teaching experiences."^ The Request for Proposal 
further defined and described these protocol materials, and the project 
staff T^as not without experience in developing materials i;*oughly sim- 
ilar to them. Cruickshank had done extensive work in creating 
simulation materials for teacher education while other members of 
the project group had been involved in various aspects of research 
in language development. This combination of experience led the group 
to propose a series of protocol materials to illustrate concepts of 
children* s oral language acquisition. (For a detailed discussion of 
protocol materials and the theoretical issues surrouiiding their devel- 
opment, see Cruickshank' s and Orlosky's papers). 

The rationale for illustrating concepts of oral language acquisition 
was rather straightforward. The staff reasoned that teachers would 
more readily accept children's oral language if given opportunities to 
observe language development and discover underlying linguistic con- 
cepts. As with most decisions, however, this one had its problems. 
Implicit in the decision was the presumption that teachers are unaware 
of the developmental implications of children's oral language even though 
immersed in it daily. Additionally, linguistic theory applicable to 
developmental interpretation of children's everyday talk has been and 
remains in a state of flux. Linguists themselves are unable to agree 
on the analysis of English ] et alone schooling and language develop- 
ment. The decision to illustrate oral language development put the 
staff irt a position of choosing among a variety of language models avail- 
able to us (tagmemic; transformational- generative in its MIT, Pennsyl- 
vania, and other forms; strat if icationa] ; and so on), or of picking and 
choosing on the basis of needs and disposition. 

These two problems presented ou^ first evaluation questions. To 
what extent are practicing teachers aware of the developmental impli- 
cations of children's oral language? 1-bw could the concepts to be 
selected be authenticated? Finally, in a technical sense, how effective 
were the materials? These questions were examined in what we labeled a 
process evaluation (or formative evaluation). 
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PROCESS EVALUATION 



In most respects the protocol materials we had chosen to develop 
bore more directly on the theoretical knowledge teachers need to inter- 
pret classroom settings rather than on samples of teaching itself. As 
such, the selection of concepts to be illustrated had to be made within 
the domain of linguistic and psychol inguistic science. A major liter- 
ature search was conducted, and a bibliography of over 200 e'ltries was 
assembled. This literature search produced. 115 major findings which, 
when organized in a taxonomy, clustered logically around eight categories. 
As might be expected, arranging our- findings in a taxonomy suggested 
several important relationships among the categories. We referred to 
these re lati "unships as a "soft theory*' which may be briefly stated as 
follows: Children process linguistic information they encounter, dis- 
covering and revising implicit rules and regularities within their 
language, passing through progressive but overlapping stages of linguistic 
complexity. Order of acquisition appears to be the only invariant presently 
observable within this process of development. 

It is important to understand that none pf the literature reviewed 
by the staff stated or implied in a direct way the conclusions we drew 
regarding this view of oral language development. While we owe a great 
debt to the intellectual work of a variety of linguists and psycho- 
linguists, our understandings and conclusions should not be taken as a 
direct reflection of current theory nor as a reflection of the individual 
•./orks we drew. upon. Responsibility rests with the project staff. 
Indeed, it is probably immodest to refer *to these relationships as a 
soft theory for we intended no contribution of this sort, but were inter- 
ested mainly in describing a coherent set of principles and assumptions 
which might give direction and dimension to our efforts. Most gener- 
ously, David Stampe, a linguist w om we consulted, referred to our 
conclusions as a soft theory, and nore out of convenience tha.u anything 
else, the designation stuck. ^ 

Clearly evident across a wide variety of studies reviewed in the 
first phase of the project was the notion that children acquired lin- 
guistic structures in a particular order. Berko's work on the acquisition 
of morphology, Carol Chomsky's on the acquisition of selected syntactic 
structures, and particularly, the rich naturalistic work of Brown and 
his associates at Harvard strongly supported the position that children 
may vary in the rate at which they acquire particular structures, but 
the order in which they do so will be invariant . 6,7 , 8, 9 , 10, 11 Variation 
of another sort stemming from socioeconomic, group, or racial status of 
children was also wel 1- document ed in the literature. 12, 13 Known tech- 
nically as sociolects or social class dialects, these variations derive 
from group membership and should be regarded as distinct from regional 
dialects. _ Finally, varying social contexts appear related to vari "ions 
in grammatical and lexical features as well as differences in field, 
mode, and style of discourse. 1^ These variations are known as registers. 

This brief summary of our exhaustive literature review highlights 
several of the major findings out of which a theoretical framework was 
created. Ultimately, from this framework, concepts were selected and 



protocol materials developed. But, before that happened, the staff 
decided that an independent authoritative review of our soft theory was 
necessary if subsequent materials were to be regarded as valid and 
authentic. 



AUTHENTICATION 

Four subject-matter consultants were invited at intervals to 
validate both the theory and later the concepts under consideration. 
Roger Shuy, Fred Williams, and Courtney Cazden each visited the project 
a^id not only suggested content revisions in the conceptual framework 
but identified language situations they felt would illustrate particular 
concepts as well. David Stampe from the Ohio State University Depart- 
ment of Linguistics reviewed and basically agreed with the soft 
theoretical account of how these concepts related to one another. Cazden 
paid a second visit to the project during the period when video tapes 
were edited and helped to remove extraneous material and sharpei. concepts 
under consideration. This phase of the process evaluation provided cor- 
rective feedback which was used to validate and sharpen tho conceptual 
or theoretical framework for the project. 

CONCEPT SELECTION 

As noted above, our literature search produced a bibliography of 
over 200 entries from which 70 were considered. The original taxonomy 
of 115 major findings was reduced to eight categories: performance- 
competence, selection-production, variabil ity, sequenti al acquisit ion, 
innateness, role, culture specific, and systematic. Reclassification 
yielded the final two categories and five subcategories. The second 
phase of the process evaluation thus completed provided essential infor- 
mation that not only facilitated the selection of concepts for video 
taping but suggested as well that these concepts comprised an important 
subject-matter for teacher education. These categories were then repro- 
duced and a panel of teachers, subject matter specialists, and teacher 
educators designated the concepts most salient to their field of activity. 
The panel was instructed to apply the following criteria to the selec- 
tion of a concept for inclusion in the taping matrix: 

Universality of acceptance 

Support of theory and/or research 

Utility 

Appl icabil ity 

Learnabil ity 

Tiinel iness 

Power ( impor t anc e ) 

Subject's lack of prior knowledge 
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9. Technical feasibility of taping 

10. Potential for obtaining natural language 

11. Potential for changing teacher behavior 

12. Interpretabil ity 

13. Current effectiveness in teacher education 

14. Fulfillment of central purpose of project 

THE CONCEPTS 

Following is a brief definition of each of the concepts selected 
for development as protocol materials. The major concepts identified 
are divided into two main categories: the "process of acquisition" 
and "variations within acquisition." 

I. THE PROCESS OF ACQUISITION 

Children progressively develop and revise sets of rules about 
their language unconsciously by drawing upon their linguistic en- 
vironment . 

A. Se quence 

i 

The process of acquisition is evident in the predictabl.e 
order of acquisition of sets of rules with variations in ratp 
of acquisition* i 

1. Concept of specific syntactic structures: 

a. Promise/tell > Structures in which "promise" is followed 
by a noun and an infinitive will be more difficult to , 
interpret than those in which "tell" is followed by a , 
noun and an infinitive. The "promise" structures depart 
from the minimal distance principle in English, which 
predicts that the noun closer to the infinitive will \ 
serve as its. subject. In the "promise" structures, the \ 
subject of "promise" also serves as the subject of the \ 
f ol lowing inf initive. 

b. Eas y/ha rd to see . In the sentence, 

Tiie clown is easy to see 

the relationship between "to see" and "clown" is that 
of verb an object rather than subject and verb. The 
. "easy to see" structure somewhat obscures this rela- 
tionship whereas in a sentence like 

The clown is eager to see 

the relationship between "clown" and "to see" is that 
of subject and verb. Consequently, the "easy/hard 
to see" structure is more likely to be misunderstood 
by young children until they learn to sort out the 
grammatical relationships of subject and verb. 



2. Concept of morphological development: 

a. Noun inflections and derived form s. Children pro- 
gressively develop and revise set^s of morphological 
rules in the following predictable order: (a) 
occasional use of a rule for an inflected form, (b) 
overgeneralization of the rule to other forms, and 
(c) refined and accurate application of the rule to 
appropriate forms. 

b. Verb inflections and adjective order .' Children pro- 
gressively develop and revise sets of morphological 
rules in the following predictable order: (a) 
occasional use of a rule for an inflected form, (b) 
overgeneralization of the rule" to other forms, and 
(c) refined and accurate application of the rule to 
appropriate forms. 

B. Compl exity 

There are developmental increases in the number as well 
as the kinds of syntactic structures produced or understood. 

1. Syntactic Acquisition 

The complexity level of a sentence is indicated by 
the number and kinds of syntactic structures used in it; 
that is, the more structures and the more kinds of struc- 
tures used in a sentence, the more complex that sentence 
is. 

2. Concept of embedding as revealed in T-unit 

The complexity level of a response can be reflected 
through T-unit word length. A longer average T-unit length 
indicates syntactic maturity. T-units by definition con- 
sist of a main clause plus all subordinate clauses. Gar- 
bles, words ivhich do not add information to the utterance, 
are deleted from T-unit word count. Aii exception to this 
are those words used to claim attention such as '*well,*' 
"see/* and "you see." 

C. Fluency 

There are progressive increases in the ease and the full- 
ness with which children speak. The fewer the nur.ber of 
hesitations, garbles, and fillers per utterance, and the 
greater the amount of language in each individual's response, 
the more fluent the speaker. 



II, THE VARIATIONS WITHLN ACQUISITIOxN 

Language acquisition involves not only the conuiion sets of 
rules developed by speakers of 'the language, but variations that 
occur within both individuals and social situations. 

O 
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A. Individual Variabi lit y^ 



Individu^^ls vary in their use of specific structures. 

1. Varicition in acquisition 

Morphology . Children of the same age differ in the kind 
and 'lumber of basic morphological structures they learn 
t^j use. While there appears to be a highly predictable 
sequence of acquisition, the rate at which children acquire 
a specific structure varies considerably. 

2. Variation in acquisition 

Derived forms and adjective order . Children of the same 
age differ in the kind and number of basic morphological 
structures they learn to use. While there appears to be 
a highly predictable sequence of acquisition, the rate at 
which given children acquire a specific structure varies 
considerably. 

B. Social Variability 

Individuals and groups develop registers of language — 
""^r language appropriate for particular social situations-- 
whi^ may vary in phonology, syntax, lexicon, and para- 
linguistics. 

PROTOCOL MATERIALS REVISION 

Before final versions of video tapes^ guide materials, and instruc- 
tor's manual were completed, prototypes of each were field tested with 
in-service teachers in a nearby school district to provide corrective 
feedback regarding the quality of each protocol episode. 

Method 

One hundred teachers from the Franklin County (Ohio) School Dis- 
trict taking a course for graduate or undergraduate credit at Ohio 
State participated in the study. They were told that all students 
would receive a passing grade in order to obtain candid opinions re- 
garding the method of instruction employed in the course. Ei^ht 3- 
hour classes were held, during which two protocol episodes were shown 
and related activities completed. After the completion of each proto- 
col episode, a questionnaire was administered to each subject. 

Procedure 

. For each protocol episode, subjects read a short introductory 
passage summarizing conclusions from research undergirding tlie concept. 
A short discussion followed. Subjects were then asked to examine the 
behavioral objectives for the episode and were allowed to raise ciues- 
tions related to the objectives. They were instruct, i how to view the 



videotaped episode, what data to record in their participant's guide, 
and how to record and summarize the data. A discussion followed each 
viewing, and summarized data from the protocol were described with 
labels, categories, generalizations, and principles. After completing 
activities in the guide materials, subjects responded to a question- 
naire. These results are summarized in Figure 1. 

RECOMMENDATIONS AND REVISIONS 

Recommendations came fiDm both participants and instructors. 
Participants 

The majority of protocol episodes were seen as both interesting 
and informative. Visually, the protocols slicited few negative re- 
actions; however, several of the episodes were plaguevi by deficiencies 
in audio quality, particularly those dealing with the promise/tell 
structure, pronominal! zation, and the easy/hard-to-see structure in 
ciiildren's language. Audio deterioration stemmed largely from tran- 
scribing these episodes from film to one-inch v^ideo tape. Some 
reduction of audio distortion was possible, however, through an atten- 
uation technique that was developed subsequent to the process evalua- 
tion and resulted in considerable improvement. 

Minor revision was recommended for one o£ the protocol episodes, 
" 'Play-Talk* in Kindergarten.'* Specifically, discussion questions were 
revised to state the ^ntent of the questions more explicitly, and two 
worksheets were combined. The protocols dealing with the acquisition 
of morphology were edited to eliminate unnecessary instances, and 
inconsistencies were eliminated. 

Instructor 

Generally, guide materials were in need of slight revision. Those 
dealing with morphology required further coordination with the video 
portions of the protocol episodes. In nearly every instance, objectives 
were considered clear, and the materials appeared to relate directly to 
the objectives. New learnings were presented at an appropriate rate, 
and, with the exception of episodes dealing with individual variations 
in the acquisition of derived form^ and adiective order, tapes and guide 
materials seemed to match the part ic ipant ' r learning capabilities. Ex- 
pository material was "easily grasped, and content was seen to be closely 
integrated with videotaped illustrations. Difficulties experienced 
with tae acquisition of s>Titax were noted by all instructors. Instructor 
ratings were basically positive regarding content, but again some tech- 
nical viifficulties were encountered. 



PRODUCT EVALUATION 

Only a painfully small sample of protocol materials aas been studied 
to date; thus, gre^t caut von should be observed in generalizing from the 
results of this and other early attempt? to evaluate the protocol notion. 



Figure 1 

Summary of Process Evaluation for Protocol Episodes 
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Questions 


Sequence of Acquisition 


Acquisition of S/ntactic 
Structures '(Pronominali zation) 


Acquisition of Syntactic 
Structures (Promis e-Tell ) 


Acquisition of Syntactic 
Structures (Minimal Distance) 


Acqui it ion of Syntactic 
Structures (Var iabi li ty) 


Acquisition of Syntactic 
Structures (Complexity) 


Acquisition of Morphological 
Structures (Verbs) 


Acquisition of Morphological 
Structures (Nouns) 


Group Variation (Register) 


Individual Variation 


Individual Variation 


Individual Variation 


Individual Variation 


Individual Variation 


1, Object! "-es were clear. 


90 


60 


93 


97 


76 


97 


80 


70 


73 


60 


57 


100 


97 


100 


2, Materials were attractive 
and interesting, ^ 


70 


30 


87 


76 


64 


89 


64 


60 


80 


47 


57 


83 


93 


73 


3, Materials built on my pre- 
vious knowledge. 


80 


53 


70 


77 


74 


89 


63 


50 


70 


50 


53 


J3 


86 


90 


4, Content was well organized. 


87 


67 


94 


87 


80 


87 


83 


60 


73 


37 


76 


90 


97 


90 


5, Learnings were clearly 
presented. 


94 


40 


86 


73 


54 


86 


67 


43 


60 


44 


53 


90 


94 


74 


6. Repetition of important 
consent was adequate. 

- — 1 


s. 


50 


95 


73 


64 


89 


63 


57 


57 


67 


53 


83 


96 


74 


7, Learning tasks were easy. 


84 


50 


87 


90 


64 


93 


80 


40 


73 


53 


46 


90 


90 


83 


8. Learninn tasks were at niy 
level of understanding. 


84 


60 


94 


87 




86 


90 


46 


76 


57 


60 


97 


90 


70 


9. Protocol was visually clear. 


60 


43 


77 


70 


76 




83 


64 


70 


80 


53 


80 


87 


70 


10. Sound track was clearly 
^ au d i b 1 e . 


63 


3 


73 


3 


57 


35 


66 


7 


54 


70 


36 


100 


57 


26 


11. CJuide materiais were easy 
to use. 


86 


7 7 


87 


90 


94 


S3 


90 


57 


63 


3^^ 


33 


100 


96 


95 


12. Ideas presented were w^rth 
learning. 


87 


43 


97 


77 


80 


93 


63 


63 


60 


43 


56 


80 


93 


,74 




Figure 1, Percent of Subjects A^rcein^ with Statementr: 



Method 



Two groups of 61 subjects each were drawn at random from a reading- 
language arts methods course ari randomly assigned to equal treatments. 
Two instructors were assigned in random order to oi equal number of 
class sessions for both conditions. The' protocol naterials condition 
was compared with a treatment in which subjects were taught the same con- 

pts through the lecture method coupled with classroom obsei> ations of 
children's oral language. Posttest moans were compared using a t test 
with significance set at the .05 level in a simple randomized design. 

Procedure 

Subjects in the protocol materials condition completed 14 episodes 
in which five concepts were illustrated in a va iety of oral language 
activities depicting aspects of the acquisition »f syntax, register, 
and morphology as well as individual difference^ in development. As 
stated previously, during each protocol segment, subjects read a short 
summary of research conclusions about the concept being illustrated. 
Following a short discussion, subjects read behavioral objectives for 
the protocol segment and asked any questions about the objectives. 
Subjects were then given directions on viewing the protocol, instruc- 
tions on how to record data in their guidebooks, and opportunities to 
work through examples before viewing the video tape. After viewing 
each tape, subjects discussed their observations. Finally, each subject 
described the data collected in terms labels, categories, geneial- 
izations and principles. 

Subjects in the lecture condition were provided with similar infor- 
mation regarding each concept illustrated in the protocol materials. 
They were also provided with written materials but hac' no opportunity 
to observe films or video tapes of children in various stages of lan- 
guage acquisition. Instead, subjects in this treatment observed 
children in public school classrooms and in informal play situations 
independent of #ie lecture class itself. 

Both groups received approximately the same number of hours of 
ini'truction. The lecture condition group received slightly over iO 
hours of instruction a;id 2 hours of observation wh" le t'.e protocol 
treatment group received 12 hcnrs of instruction. 

Instrument 

A 46- item test was constructed i"^h an equal number of items for 
all bur. one of the concepts illustrated in the protocol materials. 
Items were constructed from behavioral objectives for each protocol 
segment and judged for their concept validity by a 6-member panel from 
the project staff. Items that did not receive full acceptance by the 
panel were rewritten or rejected. Two reliabil it y coe-^ficients were 
obta ?d using the Kudor-Richardi^on methods 20 (.77) and 21 (.70). The 
stano d error of measurement for the instrument was 3.04 v/hile the 
mean item difficulty coefficient obtained was .30, and the mean item 
discrimination coefficient obtained was .28 Tests of skewness revealed 
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only a slight negatively skewed distribution (.86) while tests of 
kurtosis indicated some peakedness in the distribution (1.21). 

Results 

Posttest means between the protocol treatment (33.76; S.D.=4.71) 
and the lecture treatment (33.08; S. D. =4.40) were not significantly 
different (t=.75). 

Discussion 

Protocol materials have been suggested as a means of overcoming 
the weakness of unstructured observation and many theoretical inade- 
quacies characteristic of teacher education. Yet, protocol materials 
demoiistrated no advantage over lecture and classroom observation where 
the criterion was a written achievement test administered 7 days fol- 
lowing the completion of all r^rotocol episodes. It can be argued, and 
perhaps convincingly, that, for protocol materials, a straightforward 
achievement test lacks construct validity. If, indeed, protocol mate- 
rials have as their ultimate end improving a student's ability to 
absti 't and transform raw data from a classroom into salierH concepts 
about^ in this case, language condition, then a different sort of 
criterion measure should be contemplated. A more desirable measure 
would rt least make use of protocol-like instances of the specified 
behavior as the criterion task. This sort of measure would more nearly 
approach the logic of protocol materials as well as the sort of learner 
response anticipated in the notion. Hypothet ical ly, at least, it has 
been suggested that protocols may engender a learning- to-learn set as 
well as numerous specific learnings that may transfer readily to new 
responses or stimuli. Before firm conclusions are drawn regarding the 
effectiveness of protocol materials as medium of instruction, these 
estimates of both specific and general transfer should be obtained. 

The finding of no difference between treatments, \Aile not exactly 
what a development team would prefer, loses much of its sting when 
viewed in the context of subj ects ' perceptions of these materials. Gen- 
erally enthusiastic responses from subjects who were asked to compare 
protocols with other instructional techniques indicate that protocol 
materials are interesting and attractive, that they are well -organized 
and clear in their presentation of concepts, and that they incorporate 
a body of learnings that students consider worth learning. To the 
developer, the no-difference result should suggest evaluation procedures 
that are as interesting, precise, and clear as the materials themselves, 
and a component within the development plan that is, if not as sizable, 
at least as significant as the materials package. 
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State, served as a consultant during the project's first year. 

6. Jean Berko, "The Child's Learning of English Morphology," Word 
14 (August 1958): 150-77. 

7. Carol Chomsky, I1ie Acquisition of Syntax in Children from Five to 
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A Survey of Protocol Matei-ials Evaluation 1 
by John E, Cooper J 



A BRIEF HISTORY OF PROTOCOL MATERIALS 



There is no shor':age of instructional materials in teacher education. 
Even a cursory review of Books In Print, recent publications announced by 
book companies, and audiovisual catalogs will ijnpress the reader with the 
proliferation 'of instructional materials for teacher education. Probing 
beyond the facts of volume productions, however, will reveal that most 
of these materials ..re informational in nature.^ They address subject 
matter such as educational anthropology and child development; methodology, 
e.g., reading instruction and pupil control; concepts sucli as transfer 
of learning and motivation; operations like constructing a sociogram 
or finding the median; and educational issues such as federal aid to 
parochir.i schools and national testing. 

While there is an impressive store of informational materials in 
teacbei^ education, little can be found which is based on organized theory. 
If teaching is to rise above the level of a craft, teachers must be able 
to respond in some way other than through dc^pendence on trial v.nd error, 
common sense, and the practical. Theoretical knowledge is required for 
interpreting and solving problems. 

Concepts provide the basic elements of theoretical knowledge. Concept 
acquisition is a 5ine qua non for the exercise of expertise in any learnod 
profession. Therefore, the preparation of the professional should provide 
sufficient attention to concepts crucial to the nature of the profession. 

The need for placing fundamental concepts at the center of teacher 
education programs was emphasized in Teachers for the Real Worlcfi which 
called for the development of protocol materials to illustrate key concepts 
dra;m from psychology, sociology, and philosophy which would reproduce 
behaviors in life-like situations. In this connection. Smith differentiated 
between protocol materials and training materials. While the former deal 
with theoretical or conceptual elements, the latter address methodological 
or skill components. 

By January 1974, approximately 140 protocol products nad been 
developed as part of the Protocol Materials Project of the U. S. Office 
of Education. With the support of the National Center for the Improvement 
of Educational Systems (NCIES), approximately nine units of training 
materials had been prod'^ced at Indiana University un- r the auspices 
of the National Center for the Development of Trail., ng Materials in 
Teacher Educat ion. 
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STATEMENT OF THE PROBLEM 



The primary purpose of this report is to summarize evidence of the 
effectiveness of protocol materials in the improvement of teaching. A 
secondary objective is to place this evaluation in the context of a brief 
review of protocol materials as an educational innovation. It should 
be pointed out that the greatest proportion of funds and energies has 
been directed at training personnel in the development of protocol 
materials rather than at the evaluation of this effort. 

LIMITATIONS OF P.AST EVALUATIONS 

Even though Protocol Materials Project directors have attempt ed to 
field test their products before distribution, relatively little attention 
has been given to large-scale evaluation, dissemination, and preparation 
of trainers of preservice and in-service teachexs for using the materials. 

There are several explanations for the relative neglect of these 
essential consideratibns. Foremost is that projects are funded for a 
brief term, generally one year- This forces project directors to 
concentrate on developing materials, to the neglect of those elements 
which come after production, e.g., try-out, evaluation, training, and 
dissemination. Secondly, the resources required for executing those 
phases are not always available to protocol producers. 

METHODS OF ANALYSIS 

In reviewing^ evaluation, 73 pieces of evidence were examined, 
inventoried, and classified (see Appendix A). These items came from 
12 protocol locations. They included professional articles, manuscripts 
of addresses, summaries of studies, project reports, and letters and 
instruments for collecting data. The investigator limited his review 
to material in his jiossession at the time of the study. Some protocol 
project directors may have collected evidence of the effectiveness of 
their products that was not availdble to the investigator at the time 
this report was written » 

A matrix '*vas devised for classifying the kinds of evidence 
represented by the documents (see Figure 1 and Appendix A). Twenty- 
one of the reviewed items did not contain information directly related 
to instructional effectiveness; they are listed in Appendix B. Certain 
cells which are not applicable or which have little if any possibility 
of revealing evidence are blocked out in the matrix. 

The most powerful evidence of the effectiveness of protocol materials 
would reveal changes in the behavior of students. Since protocols are 
designed for use in teacher training rather than for the education of 
children, it is not surprising to find that to date no project has 
tried to gather this kind of evidence. To do this it would be necessary 
to demonstrate that children who were taught by teachers trained with 
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Nature of the Materials 




A. 



Films, tests, diaries, 
etc. to measure 



B. Films, panel judgments, 
anecdotal records etc. 
to measure ^2 



Test results to 
measure #3 



Questionnaires, inter- 
views, comjnentaries, etc. 
to measure 

tt. ■ 

Questionnaires, inter- 
views, commentaries, etc 
to measure -^5 



F. Letters from users, 
lists of users, testi- 
monials, etc. 

G. Professional articles, 
manuscripts of ::pecchcs, 
tapes, etc. 



K. 



H. Summaries of studies 
including field tests 



I. Project reports 
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20, 21, 22, 
44, 50 



3,4, 17,19 
20,21,22, 
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protocol materials performed significantly better than those taught 
by teachers who did not have access to these materials. It would also 
be necessary to show that there were no other significant differences 
between groups of pupils, teachers, or learning settings than the isolated 
variable of use of the protocol materials. 

A second level of evidence would show differences in teaching behavior 
resulting from teacher exposure to protocol materials. Obviously this, too, 
requires considerable effort to obtain, necessitating evidence of favorable 
differences in teacher performance before and after the introduction of 
protocols. Despite the difficulty of obtaining this information, one 
item of such evidence is revealed in this report. 

Other types of evidence of the impact of protocols are easier to 
acquire. They include concept acquisition, reactions to the -materials by 
trainees, and volume of demand for protocols. These kinds of evidence 
were collected for this report. 

FINDINGS 

Evidence of Changed Teacher Behavior 

Borg and Stone (52) * selected two of six protocol modules — '^Encourage- 
ment'* and "Extension** — developed at Utah State University during 1971-72 
i^nd tested their impact on the classroom behavior of 19 in-service 
elementary teachers of the Weber (Utah) School District. These teachers 
had received approximately 16 hours of training over a period of 2 weeks. 
Training involved reading descriptions of the concepts; completing 
practice lessons; viewing protocol films; and completing lecognition tests, 
application practice lessons, application tests, and self-practice lessons 
using audio tape recorders. 

Specific behaviors accompanying ^'Encouragement'* included general 
praise, specific praise, and use of student ideas. Those relating to 
"Extension" were prompting, seeking further clarification, refocusing, 
and redirection. 

Pre- and jx)st- training audio tapes of 20 minutes duration were made 
of each teacher performing in her classroom. These tapes were coded and 
scored by trained raters who did not know whether a particular tape was 
recorded prior to or after training. 

Virtually no difference was found in the amount of general praise 
given before and after training; however, 17 of the 19 teachers increased 
their use of specific praise. The average use of student ideas approximately 
doubled. 

Prom^^ting, which is a strategy for improving pupil response to teacher 
questions, more than doubled followin<; training. The average teacher also 
nearly doubled the use of further clarification. 

: \ 

\ 

*Nuinbers in tnis section of the report refer to those items listed ' 
in the Protocol Materials fivaluation Exhibit Inventory (see Appendix A). 



Refocusing is an attempt to enable the student to generalize his 
knowledge through questioning that locates comnicn elements drawn from 
diverse subjects. It is a relatively uncommon teaching technique. 
Eleven of the 19 subjects showed gains on this variable. 

There was no significant change in the use of redirection, a strategy 
employed to increase the number of discussion participants. This variable 
was found to be a common teaching behavior at the pretraining level. 

Evidence of Concept Acquisition n 

Utah State University tested six protocol modules dealing with 
important concepts related to teacher language (49,50,51). They 
included extension, encouragement, clarity, emphasis, feedback, 
and organisation. A criterion level calling for 80 percent mastery by 
80 percent of the subjects for each module was established. Three 
criteria were selected for evaluating each module: (a) recognition 
of teacher use of the concept on film, fb) recognition of teacher use 
of the concept in typed manuscripts of class discussions, and (c) application 
of the concept to typed transcripts of classroom discussion lessons. On 
the final field test more than 80 percent of the learners reached 
the criterion level of mastery on all 18 of the criterion measures used 
to evaluate the six modules. 

Protocol films and guides entitled "Tasks of Teaching*' have been 
produced at Michigan State University. They cover a5^sessment , goal 
setting, strategies, and evaluation. The protocol materials wove evaluated 
by using 429 Michigan State I iversity undergraduate education majors (20). 
They were randomly divided into'' an experimental group of 215 subjects 
and a control group of 214. The experimental group received instruction 
using the protocols while students in the control group received parallel 
instruction without the use of these materials. 

Two measures of concept acquisition were selected, one of concept 
recall and the other, a measure of the ability to identify the concepts 
as part of a teaching vignette. Tlie results showed the clear superiority 
in concept growth of the experimental ^^rou^,. 

Michigan State University has also been involved in the devcMpmcnt 
of protocol materials, including filmed and written aids, for respondent 
learnini;. Concept acquisition and transfer of the conccjit to a simulated 
teaching situation were tested, using iTiore tuan bOO Michigan State 
University education majors (22). Six different treatment conditions 
were established in the experiment. The results jiroJuccd strong'', evidence 
of the effectiveness of protocc)! materials on conceiU actjuisition and 
its transfer. 
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Four video vape protocols at the Far West Laboratory for Educational 
Research and Development were designed to aid in interpreting group process 
in the classroom. Acquisition tests were administered on three of the 
protocols: task roles, unifying roles, and antigroup roles to instructed 
and noninstructed groups (4,5). The results showed the ability of the 
protocols to teach concepts. Differences in; concept attainment favored 
instructed groups over the noninstructed group at the .01 level. The 
fourth protocol, stages of group growth, war not fully evaluated in the 
field test. 

Recently, protocols were completed and field tested by the Far West 
Laboratory on using student ideas, questioning, praise and corrective 
feedback, and lesson organization (6,7,8,9). Although different 
measurement techniques were employed to field test concept attainment 
for each of the four protocols, e.rly positive results of the effectiveness 
of the protocol materials were attained. 

Indiana University produced foui protocols, on film and in print, 
on cognitive interaction, affective interaction, classroom management, 
and counseling. An early evaluation report of these materials showed 
that significant learning occurred from use of each of the four units (18). 
Since this report, the Protocol Materials Project at Indiana University 
has developed and tested more thoroughly additional protocols on teacher- 
pupil interaction. Studies conducted on learning outcomes have yielded 
pre- and post- treatment data on a single group and post- treatment data 
for comparison with results of an untrained group. Results document 
significant growth on the acquisition of tested concepts (19). 

The findings of one doctoral study at Indiana University, on the 
effects of protocol and training materials on concept acquisition and 
skill acquisition on teacher trainees, suggest ''that either materials 
expressly designed as protocol mater^^als or materials expressly designed 
as training materials lead to the acquisition of both interpretive concepts 
and teaching skills. The finding that viewing protocol films instancing 
concepts about teaching behavior leads to a demonstrable acquisition of 
those behaviors (as well as the acquisition of concepts about those 
behaviors) should be of interest to future investigators^^ (19) . 

Protocol materials developed at Southern Illinois University at 
Edwardsville were in audio tape and printed form and dealt with morphological 
and syntactic features of Black Dialect. These materials were field tested 
in several states, using both education majors at the preservice level and,, 
in-service teachers (27). Twelve concept acquisition tests were administered 
on various concepts related to the linguistic content. The criterion level 
of 80 percent mastery by 80 percent of the subjects was satisfactorily 
achieved on each of the 12 tests. 

Evidence of Reaction to the Materials 

Many Protocol Materials Project directors designed methods for 
measuring the attitude of trainees to protocol materials. A smaller 
number also designed instruments to ascertain the attitude of teachers 
of trainees to ^^ro to cols. 
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User reactions to protocols falls into two broad categories: (a) 
impressions about the technical quality of the protocol, e.g., sound 
quality of a film, and (b) value judgments about the usefulness of th^ 
content for improving teaching. The most common instruments employed 
to collect both kinds of information were rating scales. 

Summaries of student reactions to the technical quality of the protocol 
are found in the reports of Utah State University (51), Michigan State 
University (20,21), Indiana University (17,19), and Southern Illinois 
University (27). 

Summaries of student reactions to the relevancy of the protocol to 
teaching can be obtained from Utah State University (51), Michigan State 
University (20,21,22), the Far West Laboratory (5,6,7,8,9), Indiana 
University (17,19), and Southern Illinois University (27). 

Instructor reactions were gathered at Indiana University (17) and 
Southern Illinois University (27). Six instructors from several institu- 
tions tried out Indiana University protocols and gave positive responses 
to their quality, appi*opr lateness of content, and utility in promoting 
intended concepts. 

Evaluation of the Southern Illinois University audio tapes in Black 
Dialect by eight specialists in fields such as speech, linguistics, and ♦ 
anthropology appears in narrative form (27). 

Evidence of Demand for the Materials 

The Protocol Materials Project director at the University of Colorado 
is giving considerable attention to the subject of dissemination of protocols 
produced at that institution. The decision to publish and disseminate 
University of Colorado protocol materials was made in April 1973. Ten 
thousand brochures describing the materials were printed with the first 
mailing on May* 1, 1973 (46). By Novemoer 15, 1975, more than 300 requests 
had been received for previewing, renting, or purchasing or for further 
information about the materials (45,48). 

SUMMARY OF THE FINDINGS AND RECONU^ENDATIONS 

A review of research on the effectiveness of protocol materials in 
improving teaching and learning shows that no attempt has been made as 
yet to discover the influence, if any, on the behavior of pupils. One 
study reveals that protocols result in favorable changes of teaching 
behavior. Positive results have been obtained on the acquisition of 
concepts by preservice and in-service teachers. There is also evidence 
of the reactions of both trainees and their teachers to the technical 
qualities and relevance of protocol materials. IVhilc little attention 
has been directed to dissemination, there is a t;rowini; evideu^e of 
demand for protocol materials by preservice and in-service educators. 
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The following recommendations relate to future evaluations of 
protocol materials: 



1. As theory is related to practice^ so are concepts .related to 
skills. Concerted attention should be given to identifying and^ searching 
for relationships between instrut.tional concepts and skills which 
influence teaching and learning. The work of Hudgins in cataloging 
concepts and Turner in cataloging skills should be supported as an 

• important contribution to this work. 

2. Protocol and training products are essential instructional 
materials for preservice and in-service competency-based teacher 
education programs. In view of the national transition to competency- 
based teacher education, financial support ii> needed to assist the 
development of these materials. 

5. Ultimately, the question of whether or not protocol and training 
materials used in competency-based teacher education programs make any 
difference to pupil performance should be researched. This question cannot 
be answered finally and fairly, however, until materials have been developed 
which exemplify the full range of essential concepts and skills, until 
teacher educators have been properly trained in their use, and until 
trainees have completed competency-based programs that have relied on 
these tools. 

4. In the meantime^ the field testing and consequent revision of 
new protocol materials should be encouraged. IVhile collecting user 
reactions to the materials serves a legitimate purpose, it cannot take 
the place of tests of concept attainment. These should be conducted as 
realistically a? possible. Audio- and video-taped evidence of trainee 
learning, when feasible, is superior to the evidence produced by paper- 
and-pencil tests. Similarly, information gathere^d from responses to 
filmed testing should be a more accurate indicator of conceptual 
power than information collected from a written simulation. 
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NOTES 



David Gliessman, "What Are Protocols: Their Nature and Purpose?" 
in Handbook on the Developinent and Use of Protocol Materials for 
Teacher Education (Chipley, Fla. : Panhandle Area Educational 
Cooperative, 1973) . 

B. Othanel Smith and others. Teachers for the Real World (Washington, 
D. C. : American Association of Colleges for Teacher Education, 1969]. 
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Protocol Materials Evaluation Exhibit 
Inventory 



Source 



Number 



Classif icat ion Title 



Bucicrell University 
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Protocols in Developmental 
Reading: May 1970, September 
1973 



California State 
University at North- 
ridge 



SF 



Letter to John Cooper, 
Dec. 12, 1973 



Far West Laboratory 



3C, 3H, 3L Protocols on Group Process, 

4D, 4H Instructor's Manual 



3C, 3H, 3L Learning Concepts a^out 

4D, 4H, 4L Group Process: An ytvaluatioii 

of Protocol Materials 

3C, 3G, 3L 'The Group Process Protocols: 

40, 4G, 4L -The 1971--72 Protocol Project 

Report for AERA Meeting 
Feb. 25-Mar, 2, 1973 

3C, 3K, 5L Lesson Organization: Protocol 

40, 4K, 4L Materials for Teachers, 1973 

3C, 3K, 5L Praise and Corrective Feed- 

4D, 4K, 4L back: Protocol Materials 

for Teachers, 1973 



Florida Department 
of Education 



10 



3C, 3K, 3L Questioning: Protocol Mater- 

40, 4K, 4L ials for Teachers, 1973 

3C, 3K, 3L Using Student Ideas: Proto- 

4D, 4K, 4L col Materials for Teachers, 1973 

4L Student Analysis Form for 

Field Trial Evaluation of 
Protocol Material s for 
Teacher Education 



11 



5L 



Analysis Form for Instruction 
and Specialists #or Field 
Trial Evaluation of Protocol 
Materials for Teacher Hducn- 
t i on 
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Source 

Florida Department 
of Education 



Indiana University 



Michigan State 
University 



. Number 



12 



13 

14 

IS 
16 

17 
18 



19 



20 



21 



22 



23 



Classification •title i 

4L Trainee Analys^Y? Form for 

^Protocol Materials for 
Teacher Education 

5L Instructor's Evaluation Forjn: ' 

V. Concepts and Patterns in 

Teacher-Pupil Interaction 

4L Student's Evaluation Form: 

Concepts and Patterns in 
Teacher- Pupil Interaction 

Inventory IC (Revi^^ion 5/17/73) 

3L Inventory D, Part I (Revision 

9/17/73} 

4D, 4H, 4L R^^^ction to Protocol Materials: 

5H A Survey of Students and 

Faculty Users 

3C, 3V;, 3L A Preliminary Evaluation 

4f"„ 4H, 4L Report on the Development 

and Use of Filmed Protocol 
Materials within Two Instructional 
Strategies 

3C, 3H An Evaluation Summary and 

40, 4H Dissertation Abstract on 

the Effectiveness of Protocol 

Materials 

3C, 3H Experimental and Field 

4D, 4H Evaluation of Protocol Mater- 

ials Developed To Teach 
''Tasks of Teaching'' Concepts, 
•. f Report' #2 

3C, 3H University of South Florida 

4D, 4H, 4L Field Test of the Michigan 

State University Protocol 
Materials on Learning, Jan, 
1973 

3C, 3H, 3L MSU Research and Evaluation, 

4D, 4H, .4L Report #1, May 1972 



3K 



Carrel Lesson One: 
Teaching 



The Tasks 
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Source 



Number Classification Title 



Michigan State 24 
University 



Ohio State University 25 



3K 



4D, 4H 



Education 200, Unit IV, 
Teaching Task ^3: The 
Process of Strr.tegy Selection 

A Protocol Material s 
Evaluat ion : The Language 
of Children 



Southern Illinois 
University 



26 



27 



4D, 4H 



3C, 51 
4D, 41, 4L 
5F, 51 



Field Trial Report: The 
Language of Children 

Final Report : Protocol 
Materials Development Pro- 
ject, SIU at Edwardsville- 



5K, 5L 



Identifying the Morphologi- 
cal and Syntactic Features 
of Black Dialect 



29 



6F 



Telephone conversation with 
Theresa Love, Protocol 
Material s Project Director, 
SIU, January 2, 1974 



SUNT at Buffalo 



4L 



Field Test Evaluation Fomis, 
Fredonia, N.Y. 



Teaching Research 



4L, 5L 



Protocol Materials for 
Teacher Education, Learner 
Outcomes, Field Trial 
Evaluation Guide, March 1971 



5K, 4L 



Untitled (Important Defini- 
tion] 



University of Colorado 53 



3L 



Student Background Information 
and Questi onnaircs for 
Concepts about Teaching 



34 



5L 



I nst ructor Eval uation Ques- 
tionnaire for Conccptua 1 i i ng 
t he Process of Inst rue t ion 



55 



4L 



S tuUent Eva 1 uat i on (^i cs t i on - 
nai re for Concept u:i I i zing the 
Process of Instruct ion 



56 



I nstruc tor Evaluat ion-Qiies- 
t ionnai rc for Learners and 
'nici r Characteri st i cs 
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Source Number Classification Title 



University of Colorado 37 4L Student Evaluation Question- 

naire for Learners and Their 
Characteristics 

38 5L Instructor Evaluation Ques- 

tionnaire for Verbal Inter- 
action in the Cognitive 
Dimension 

39 4L Student Evaluation Question- 

naire for Verbal Interaction 
in the Cognitive Dimension 

40 5L Instructor Evaluation Ques- 

tionnaire for Organizing Facts 
To Teach Meaningful Relation- 
ships 

41 4L Student Evaluation Question- 

naire for Organizing Facts 

To Teach Meaningful Relationships 

42 5L Instructor Evaluation Ques- 

tionnaire for Pair Verbal 
Behavior 

43 4L Student Evaluation Question- 

naire for Fair Verbal Behavior 

' 44 3C, 3H, 3L evaluation Report of the 

4D, 4H, 4L 1970-72 Protocol Materials 

5E, 5H, 5L Units Developed by the Pro- 

tocols Materials Development 
Project, University of 
Colorado 

45 6F Dissemination Report: List 

of Person? Ordering Material 
for Preview, Rental or Sale, 
August 1-November 15, 1973 

46 61 The Dissemination of Protocol 

Materials: One Project's 
Answer 

47 6F ' - Letters from Users of Univer- 

sity of Colorado Protocol 
/ Materials 
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Source 



Number 



Classification Title 



University of Colorado 48 



Utah State University 49 



6G 



3G, 4G 



The University of Colorado 
Protocol Project: A Case 
Study 

Protocols: Competency Based 
Teacher Education Modules, 
By Walter Borg in E ducational 
Technology 12, no. 10 (October 
T973) 



50 



3C, 3H 
4D, 4H 



Field Testing and Evalua- 
tion in the Utah State 
University Protocol Project 



51 



5C, 31 The USU Protocol Project; 

4D, 41, 4L Final Report, 1971-72 
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2G, 2H 



What are Protocol Materials? 
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ERIC 



Source 



Materials Reviewed 
Number 



Far West Laboratory 



Florida Department 
of Education 



Indiana University 



Michigan State 
University 

Southern Illinois 
University 

SUNY at Buffalo 



Teach iniz Research 



53 
54 

55 

56 

57 

58 
59 

60 

61 
62 

63 

64 
65 

66 

67 
68 



but Not Applicable to the Study 
Title 

Introduction to Protocols 

Field Test: Protocol Materials on Group 
Process 

Florida Protocol Materials Project, Sept. 19, 
20, 21, 22 

(Untitled paper) Review of 1971-72 Activities 
and a Summary of 1972-73 Activities 

Memo; Subject: Agenda Item for LTI and 
Directors Meeting in Tampa, January 25-25, '73 

Protocol Materials Review Inventory 

Protocol Materials Review Inventory and 
Written Material s Form 

Concepts and Patterns in Teacher- Pupil 
Interaction: Categorizing Classroom Behavior, 
Filmed Version 

Categorizing Teacher Behavior, Part 1 

Protocol Materials Evaluation Plan for 
Michigan State University, 1973-74 /■ 

Protocol Materials Development Project: 
Notes for Instructors 

Project in Ethnography in Education 

Project in Ethnography in Education TraLning. 
Materials: A Description 

Some Specifics on Field Testing and Training 
Activi ties 

Progress Report, March 21, 197 3 

Protocol Materials for Teacher . [:du(?^at ion: 
Learner Outcomes User^s (lUiJe, Aj^r 1 1 11)71 
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Source 



Number Title 



University of Colorado 69 

70 

71 

72 
73 



Progress Report: Development of Protocols 
of Social Science Concepts and Proposed 
Design for Testing and Evaluation of the 
197 3 Products 

Field Testing, Evaluation, Revision and 
Dissemination of Protocol Materials Produced 
during 1970-72 by the Protocol Materials 
Development Project, University of Colorado 

Protocol Materials Development Project: 
A Summary Report, 1970-1973 

Instructor Background Information 

Protocol Materials Devel opment Project : 
A Summary Report, 1970-1972 
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ABOUT EK'C 




The Educational Resourc\s Information Center (ERIC) forms a nation- 
wide information system established by the U.S. Office of Education, 
designed to serve and advanc ©^American education. Its basic objective is 
to provide ideas ant' information on significant current documents (e.g., 
research reports, articles, theoretical papers, program descriptions, 
published and unpublished conference papers, newsletters, and curriculum 
guides or studies) and to publicize the availability of such documents. 
Central ERIC is the term given to the function of the U.S. Office of Edu- 
cation, which provides policy, coordination, training funds, and general 
services to the clearinghouses in the information system. Each clear- 
inghouse focuses its activities on a separate subject-matter area; acquires, 
evaluates, abstracts, and indexes documents; processes many significant 
documents into the ERIC system; and publicizes available ideas and infor- 
mation to the education community through its own publications, those of 
Central ERIC, and other educational media. 

TEACHER EDUCATION AND ERIC 

The ERIC Clearinghouse on Teacher Education, established June 20, 
1968, is sponsored by three professional groups-- the American Association 
of Colleges for Teacher Educ'S^tion (fiscal agent); the Association of 
Teacher Educators; and. Instruction and Professional Development , National 
Education Association. It is located at One Dupont Circle, Washington, 
D.C. 20056. 

SCOPE OF CLEARINGHOUSE ACTIVITIES 

Users of this guide are encouraged to send to the ERIC Clearinghouse 
on Teacher Education documents related to its scope, a statement uf which 
fol lows : 

The Clearinghouse is responsible for research reports, curriculum 
descriptions, theoretical papers, addresses, and other materials 
relative to the preparation of ochool per sonnel (nursery , elemen- 
tary, secondary, and supporting school personnel) ; the preparation 
and development of teacher educators; the profession of teaching; 
and the fields of health, physical education , and recreation , The 
scope includes the preparation and continuing development of all 
instructional personnel , their functions and roles. While the 
major interest of the Clearinghouse is professional preparation 
and practice in America, it also is interested in international 
aspects of the field. 

The scope also guides the Clearinghouse's Advisory and Policy Council 
and staff in decision making relative to the commissioni n^.\ of monographs, 
bibliographies, and directories. The scope is a flexible Quidc in the 
idea and information needs of those concerned with pre- and in-service 
preparation of school personnel and the profession of teaching. 
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